2024-09-12T02:28:43.750 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T02:28:43.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:28:43.867 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901632' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpZfwhPmO0DF73YS+OEwFUXIemwBcWcJFugj/42dDcm5zDojY1pqGybUXdpnNLYOqGxEjApiiXzCGSuHoZWkOA= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOw83Si6nG/65e0e0mqspQEsiulnECI+XyfDlxgjAh+IvfIcbKJvHkZQnr3mc1yMuJprHU2KvgqMWVrvQwLtsN4= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGx7wOkid+oFGtQlHimBc9zPMlu44+zxSEDAcNkof+ZAVl7ufKjoGEVBG+drwUB3kVeFiDMPaiIOneDRN5m1Ls= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T02:28:43.867 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T02:28:43.868 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T02:28:43.868 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T02:28:43.870 INFO:teuthology.task.internal:Checking packages... 2024-09-12T02:28:43.892 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T02:28:43.892 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T02:28:43.892 INFO:teuthology.packaging:ref: None 2024-09-12T02:28:43.892 INFO:teuthology.packaging:tag: None 2024-09-12T02:28:43.892 INFO:teuthology.packaging:branch: squid 2024-09-12T02:28:43.892 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:28:43.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T02:28:44.121 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T02:28:44.122 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T02:28:44.129 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T02:28:44.130 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T02:28:44.142 INFO:teuthology.task.internal:Saving configuration 2024-09-12T02:28:44.162 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T02:28:44.171 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T02:28:44.197 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:23:43.751303', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpZfwhPmO0DF73YS+OEwFUXIemwBcWcJFugj/42dDcm5zDojY1pqGybUXdpnNLYOqGxEjApiiXzCGSuHoZWkOA='} 2024-09-12T02:28:44.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:23:43.749747', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOw83Si6nG/65e0e0mqspQEsiulnECI+XyfDlxgjAh+IvfIcbKJvHkZQnr3mc1yMuJprHU2KvgqMWVrvQwLtsN4='} 2024-09-12T02:28:44.237 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:23:43.747919', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGx7wOkid+oFGtQlHimBc9zPMlu44+zxSEDAcNkof+ZAVl7ufKjoGEVBG+drwUB3kVeFiDMPaiIOneDRN5m1Ls='} 2024-09-12T02:28:44.238 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T02:28:44.244 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T02:28:44.245 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T02:28:44.245 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T02:28:44.245 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T02:28:44.326 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3666bbf520>, signals=[15]) 2024-09-12T02:28:44.326 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T02:28:44.333 INFO:teuthology.task.internal:Opening connections... 2024-09-12T02:28:44.333 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:28:44.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:28:44.408 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-12T02:28:44.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:28:44.481 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-12T02:28:44.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:28:44.556 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T02:28:44.562 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-12T02:28:44.580 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-12T02:28:44.580 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-12T02:28:44.637 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:28:44.638 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:28:44.639 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:28:44.639 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-12T02:28:44.662 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-12T02:28:44.680 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-12T02:28:44.681 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:28:44.738 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:28:44.739 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:28:44.740 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-12T02:28:44.762 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-12T02:28:44.780 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-12T02:28:44.781 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-12T02:28:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-12T02:28:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-12T02:28:44.838 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:28:44.839 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:28:44.840 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:28:44.840 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:28:44.840 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-12T02:28:44.861 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T02:28:44.873 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T02:28:44.879 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T02:28:44.879 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:28:44.882 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:28:44.884 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:28:44.901 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T02:28:44.907 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T02:28:44.907 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:28:44.940 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:28:44.943 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:28:44.974 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T02:28:45.051 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T02:28:45.066 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T02:28:45.066 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T02:28:45.066 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:28:45.067 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:28:45.067 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:28:45.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:28:45.085 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-12T02:28:45.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:28:45.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:28:45.086 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-12T02:28:45.087 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-12T02:28:45.142 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:28:45.142 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-503.el9.x86_64 2024-09-12T02:28:45.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-09-12T02:28:45.144 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:28:45.144 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-503.el9.x86_64 2024-09-12T02:28:45.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-09-12T02:28:45.145 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:28:45.146 INFO:teuthology.task.kernel:Running kernel on smithi145: 5.14.0-503.el9.x86_64 2024-09-12T02:28:45.146 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2024-09-12T02:28:47.789 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.2 MB 00:00 2024-09-12T02:28:49.568 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.2 MB 00:02 2024-09-12T02:28:49.625 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.2 MB 00:02 2024-09-12T02:28:50.565 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-12T02:28:51.727 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-12T02:28:55.007 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-12T02:28:55.825 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:05 2024-09-12T02:28:56.386 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.5 MB 00:00 2024-09-12T02:28:56.524 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-12T02:28:57.404 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T02:28:57.987 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-12T02:28:58.991 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-12T02:29:00.459 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.5 MB 00:00 2024-09-12T02:29:01.917 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-12T02:29:02.830 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T02:29:02.876 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T02:29:04.622 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:29:04.890 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:29:04.890 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:29:04.991 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-12T02:29:04.997 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-12T02:29:04.997 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-09-12T02:29:04.997 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-12T02:29:04.997 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-12T02:29:04.997 INFO:teuthology.orchestra.run.smithi145.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout:Install 4 Packages 2024-09-12T02:29:04.998 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:29:04.999 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 79 M 2024-09-12T02:29:04.999 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 126 M 2024-09-12T02:29:04.999 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-12T02:29:05.202 INFO:teuthology.orchestra.run.smithi145.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 224 kB/s | 23 kB 00:00 2024-09-12T02:29:05.577 INFO:teuthology.orchestra.run.smithi145.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-12T02:29:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 37 MB/s | 29 MB 00:00 2024-09-12T02:29:06.278 INFO:teuthology.orchestra.run.smithi145.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-12T02:29:06.278 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:29:06.278 INFO:teuthology.orchestra.run.smithi145.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-12T02:29:06.728 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:29:06.728 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:29:06.812 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T02:29:06.816 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T02:29:06.816 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-09-12T02:29:06.817 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:29:06.818 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 79 M 2024-09-12T02:29:06.818 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 126 M 2024-09-12T02:29:06.818 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T02:29:06.963 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-12T02:29:07.330 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-12T02:29:07.330 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-12T02:29:07.473 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 41 kB/s | 23 kB 00:00 2024-09-12T02:29:08.202 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T02:29:08.299 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-12T02:29:08.302 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-12T02:29:08.303 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-12T02:29:08.957 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-12T02:29:09.692 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-12T02:29:09.692 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:29:09.692 INFO:teuthology.orchestra.run.smithi046.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-12T02:29:10.208 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:29:10.209 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:29:10.296 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-12T02:29:10.304 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-12T02:29:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-12T02:29:10.306 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-09-12T02:29:10.306 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:29:10.306 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 79 M 2024-09-12T02:29:10.306 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 126 M 2024-09-12T02:29:10.306 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-12T02:29:10.400 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T02:29:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 245 kB/s | 23 kB 00:00 2024-09-12T02:29:10.761 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-12T02:29:10.763 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T02:29:10.763 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T02:29:10.987 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-12T02:29:11.395 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 35 MB/s | 35 MB 00:00 2024-09-12T02:29:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:29:11.595 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:11.654 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-12T02:29:11.654 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:29:11.655 INFO:teuthology.orchestra.run.smithi196.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-12T02:29:11.760 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T02:29:11.761 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T02:29:12.332 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-12T02:29:12.691 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-12T02:29:12.691 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-12T02:29:13.696 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-12T02:29:13.696 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-12T02:29:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T02:29:14.403 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:14.549 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:29:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:16.152 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-12T02:29:16.847 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:29:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:18.010 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:19.371 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:19.466 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:29:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:29:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:23.177 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:29:24.300 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:29:24.671 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:29:24.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:29:28.312 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:29:29.553 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:02.627 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:02.666 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:03.045 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:03.046 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:30:03.046 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:30:03.046 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:30:03.901 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:03.901 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:30:03.901 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:30:03.902 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-12T02:30:04.251 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:04.336 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:04.336 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:04.337 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T02:30:04.337 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T02:30:04.337 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T02:30:04.337 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T02:30:04.337 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T02:30:04.337 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:04.396 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:04.397 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:04.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.06 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 82.el9 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Group : Unspecified 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 8274670 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2024-09-12T02:30:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : builder@centos.org 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : CentOS 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:30:04.430 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:30:04.431 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:30:04.431 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 5.14.0-505.el9.x86_64 2024-09-12T02:30:04.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:30:05.047 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-12T02:30:06.318 INFO:teuthology.orchestra.run.smithi145.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:30:06.348 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-12T02:30:06.350 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-12T02:30:06.368 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.IPqKMrDw9G 2024-09-12T02:30:06.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IPqKMrDw9G 2024-09-12T02:30:06.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.IPqKMrDw9G 2024-09-12T02:30:06.585 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.IPqKMrDw9G is 6KB 2024-09-12T02:30:06.633 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.IPqKMrDw9G 2024-09-12T02:30:06.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:30:06.719 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:30:06.720 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:30:06.720 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:30:06.720 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T02:30:06.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T02:30:06.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-09-12T02:30:07.622 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:07.664 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:07.940 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:08.059 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:08.059 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:30:08.059 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:30:08.059 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:30:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:30:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:30:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:30:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:08.937 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:30:08.937 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.083 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:30:09.084 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-12T02:30:09.290 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:09.373 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.373 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.376 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.376 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T02:30:09.377 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T02:30:09.377 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T02:30:09.377 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T02:30:09.377 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:09.423 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:09.435 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.435 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-09-12T02:30:09.466 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-09-12T02:30:09.466 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-09-12T02:30:09.466 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 82.el9 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8274670 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-09-12T02:30:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:30:09.468 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:30:09.469 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:30:09.469 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:30:09.508 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.509 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.509 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.509 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T02:30:09.509 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T02:30:09.509 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-12T02:30:09.509 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T02:30:09.509 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:30:09.568 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:30:09.568 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-09-12T02:30:09.610 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-09-12T02:30:09.610 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 82.el9 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8274670 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:30:09.611 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:30:09.612 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:30:09.613 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:30:09.613 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-505.el9.x86_64 2024-09-12T02:30:09.613 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:30:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-12T02:30:10.283 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-09-12T02:30:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:30:11.367 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-12T02:30:11.369 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-12T02:30:11.384 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.UQclS2Fa4m 2024-09-12T02:30:11.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UQclS2Fa4m 2024-09-12T02:30:11.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.UQclS2Fa4m 2024-09-12T02:30:11.598 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.UQclS2Fa4m is 6KB 2024-09-12T02:30:11.620 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:30:11.637 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-09-12T02:30:11.639 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-12T02:30:11.647 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.UQclS2Fa4m 2024-09-12T02:30:11.654 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.MTTx6woTLv 2024-09-12T02:30:11.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MTTx6woTLv 2024-09-12T02:30:11.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:30:11.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.MTTx6woTLv 2024-09-12T02:30:11.725 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:30:11.725 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:30:11.725 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:30:11.725 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T02:30:11.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T02:30:11.861 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.MTTx6woTLv is 6KB 2024-09-12T02:30:11.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-12T02:30:11.910 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.MTTx6woTLv 2024-09-12T02:30:11.925 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:30:11.987 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:30:11.987 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:30:11.987 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:30:11.987 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T02:30:11.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T02:30:12.122 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-09-12T02:30:36.886 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T02:30:36.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-12T02:30:36.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:30:41.883 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T02:30:41.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:30:41.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:30:42.125 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-12T02:30:42.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-12T02:30:42.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:00.413 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T02:31:09.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:31:09.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:12.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T02:31:24.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:31:24.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:27.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T02:31:36.901 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T02:31:42.133 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T02:31:42.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:31:42.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:45.629 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-12T02:31:45.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-12T02:31:45.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:46.959 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-12T02:31:47.494 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-12T02:31:47.494 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:31:47.495 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-12T02:31:47.509 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:31:47.510 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:31:47.510 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:31:47.510 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:31:48.510 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T02:31:48.511 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T02:31:48.606 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-09-12T02:31:48.628 DEBUG:teuthology.parallel:result is None 2024-09-12T02:31:51.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-12T02:31:51.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:31:52.480 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-12T02:31:53.023 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-12T02:31:53.023 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:31:53.024 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-12T02:31:53.039 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:31:53.039 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:31:53.039 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:31:53.039 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:31:54.041 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-12T02:31:54.041 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T02:31:54.225 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-09-12T02:31:54.251 DEBUG:teuthology.parallel:result is None 2024-09-12T02:32:03.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:32:03.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:33:03.634 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T02:33:24.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:33:24.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:33:24.940 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-12T02:33:25.306 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:33:25.306 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:33:25.306 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-12T02:33:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:33:25.324 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:33:25.324 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:33:25.324 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:33:26.325 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T02:33:26.325 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T02:33:26.425 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-09-12T02:33:26.445 DEBUG:teuthology.parallel:result is None 2024-09-12T02:33:26.445 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T02:33:26.452 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T02:33:26.452 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:33:26.455 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:33:26.458 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:33:26.479 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T02:33:26.486 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T02:33:26.496 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T02:33:26.497 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:33:26.514 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:33:26.519 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:33:26.617 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T02:33:26.623 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T02:33:26.624 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:33:26.626 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:33:26.628 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:33:26.657 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.667 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.669 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.679 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.695 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.708 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:33:26.709 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T02:33:26.716 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T02:33:26.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:33:26.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:33:26.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:33:26.752 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T02:33:26.761 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T02:33:26.761 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:33:26.788 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:33:26.793 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:33:26.809 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:26.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:26.935 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.061 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:33:27.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:33:27.129 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:27.176 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:27.256 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.344 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:33:27.345 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:33:27.411 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:27.457 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:33:27.515 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:33:27.643 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:33:27.643 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:33:27.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-12T02:33:27.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-12T02:33:27.714 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-12T02:33:27.754 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:33:27.766 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:33:27.806 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:33:28.262 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T02:33:28.269 INFO:teuthology.task.internal:Starting timer... 2024-09-12T02:33:28.270 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T02:33:28.280 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T02:33:28.289 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T02:33:28.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-09-12T02:33:28.337 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-09-12T02:33:28.339 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-09-12T02:33:28.398 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-09-12T02:33:28.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-12T02:33:28.460 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-12T02:33:28.462 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T02:33:28.463 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-09-12T02:33:28.487 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-09-12T02:33:28.487 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-09-12T02:33:28.511 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-09-12T02:33:28.512 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-12T02:33:28.550 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-12T02:33:28.550 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-12T02:33:28.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:33:28.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:33:28.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:33:28.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:33:28.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:33:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:33:28.638 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T02:33:28.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:33:28.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:33:28.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:33:28.749 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T02:33:28.758 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T02:33:28.773 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T02:33:28.774 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi145.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T02:37:16.135 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-12T02:37:16.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:37:16.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:37:16.215 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-12T02:37:16.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-12T02:37:16.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-12T02:37:16.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:37:16.368 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-12T02:37:16.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-12T02:37:16.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-12T02:37:16.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:37:16.525 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-12T02:37:16.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-12T02:37:16.607 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T02:37:16.622 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T02:37:16.623 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:37:16.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:37:16.626 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:37:16.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:37:16.628 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:37:16.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:37:16.657 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:37:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:37:16.672 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:37:16.675 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:37:16.680 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:37:16.692 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:37:16.696 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-09-12T02:37:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-09-12T02:37:16.708 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-09-12T02:37:16.715 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-09-12T02:37:16.718 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-12T02:37:16.722 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:37:16.730 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-12T02:37:16.731 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:37:16.736 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:37:16.744 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:37:16.747 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:37:16.758 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:37:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.794 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:37:16.806 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:37:16.806 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-12T02:37:16.806 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.807 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.807 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.807 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:37:16.818 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:37:16.818 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-12T02:37:16.818 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.819 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.819 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.819 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:37:16.820 INFO:teuthology.run_tasks:Running task install... 2024-09-12T02:37:16.835 DEBUG:teuthology.task.install:project ceph 2024-09-12T02:37:16.835 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T02:37:16.835 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T02:37:16.835 INFO:teuthology.task.install:Using flavor: default 2024-09-12T02:37:16.840 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T02:37:16.840 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T02:37:16.841 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:37:16.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:37:16.843 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:37:16.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:37:16.844 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:37:16.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:37:16.994 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:37:16.994 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:37:16.998 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:37:16.998 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:37:17.001 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:37:17.001 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:37:17.105 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:37:17.106 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:37:17.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:37:17.112 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:37:17.112 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:37:17.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:37:17.116 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:37:17.116 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:37:17.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:37:17.132 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:37:17.132 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:37:17.137 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:37:17.137 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:37:17.144 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:37:17.144 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:37:17.198 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:37:17.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:37:17.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:37:17.273 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:37:17.278 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:37:17.289 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:37:17.337 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-12T02:37:17.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-12T02:37:17.343 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-09-12T02:37:17.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-09-12T02:37:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-09-12T02:37:17.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-09-12T02:37:17.574 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-12T02:37:17.585 INFO:teuthology.orchestra.run.smithi145.stdout:45 files removed 2024-09-12T02:37:17.597 INFO:teuthology.orchestra.run.smithi046.stdout:45 files removed 2024-09-12T02:37:17.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:37:17.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:37:17.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:37:18.166 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-12T02:37:18.169 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-12T02:37:18.199 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-12T02:37:18.442 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-12T02:37:18.453 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T02:37:18.491 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T02:37:18.709 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T02:37:18.729 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T02:37:18.767 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T02:37:19.234 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-12T02:37:19.475 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-12T02:37:20.529 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-12T02:37:21.410 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-12T02:37:22.068 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-12T02:37:22.439 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-12T02:37:26.536 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-12T02:37:26.881 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T02:37:26.952 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T02:37:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 19 kB 00:00 2024-09-12T02:37:28.457 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-12T02:37:28.751 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T02:37:29.145 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-12T02:37:29.236 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T02:37:30.024 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T02:37:34.408 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T02:37:34.684 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-12T02:37:35.463 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-12T02:37:36.827 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:36.828 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:36.840 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:37:36.841 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:37:36.842 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:37:36.843 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:37:36.845 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:37:36.847 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:37:36.848 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:37:36.917 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-12T02:37:36.931 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:37:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:37:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:37:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-12T02:37:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:37:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:37:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:37:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:37:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:37:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:37:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:37:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:37:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 8 Packages 2024-09-12T02:37:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:37:36.945 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 226 M 2024-09-12T02:37:36.945 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-12T02:37:37.077 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.078 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.089 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:37:37.090 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.091 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:37:37.092 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.094 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:37:37.096 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.097 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.159 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:37:37.180 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:37:37.181 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:37:37.182 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-09-12T02:37:37.183 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:37:37.184 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:37:37.185 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:37:37.186 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:37:37.187 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:37:37.188 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:37:37.189 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:37:37.190 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:37:37.191 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-12T02:37:37.192 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================== 2024-09-12T02:37:37.193 INFO:teuthology.orchestra.run.smithi145.stdout:Install 148 Packages 2024-09-12T02:37:37.193 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 8 Packages 2024-09-12T02:37:37.193 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:37:37.193 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 226 M 2024-09-12T02:37:37.193 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-12T02:37:37.276 INFO:teuthology.orchestra.run.smithi046.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-12T02:37:37.501 INFO:teuthology.orchestra.run.smithi046.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 897 kB 00:00 2024-09-12T02:37:37.593 INFO:teuthology.orchestra.run.smithi046.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T02:37:37.685 INFO:teuthology.orchestra.run.smithi046.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-12T02:37:37.777 INFO:teuthology.orchestra.run.smithi046.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-12T02:37:37.815 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.816 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.827 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:37:37.827 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:37:37.828 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:37:37.829 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.831 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:37:37.833 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.834 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:37:37.896 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-12T02:37:37.901 INFO:teuthology.orchestra.run.smithi145.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:37:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:37:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:37:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:37:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:37:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:37:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:37:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:37:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:37:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:37:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:37:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:37:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-09-12T02:37:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:37:37.923 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 226 M 2024-09-12T02:37:37.923 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-12T02:37:37.935 INFO:teuthology.orchestra.run.smithi046.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 2.4 MB 00:00 2024-09-12T02:37:38.019 INFO:teuthology.orchestra.run.smithi046.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-12T02:37:38.236 INFO:teuthology.orchestra.run.smithi046.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 21 MB 00:01 2024-09-12T02:37:38.303 INFO:teuthology.orchestra.run.smithi046.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-12T02:37:38.368 INFO:teuthology.orchestra.run.smithi145.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-09-12T02:37:38.594 INFO:teuthology.orchestra.run.smithi145.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.3 MB/s | 897 kB 00:00 2024-09-12T02:37:38.740 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-12T02:37:38.836 INFO:teuthology.orchestra.run.smithi145.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 311 kB/s | 145 kB 00:00 2024-09-12T02:37:38.861 INFO:teuthology.orchestra.run.smithi046.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-12T02:37:38.953 INFO:teuthology.orchestra.run.smithi046.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 228 kB 00:00 2024-09-12T02:37:39.011 INFO:teuthology.orchestra.run.smithi145.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-12T02:37:39.028 INFO:teuthology.orchestra.run.smithi046.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 361 kB/s | 27 kB 00:00 2024-09-12T02:37:39.112 INFO:teuthology.orchestra.run.smithi046.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 380 kB/s | 32 kB 00:00 2024-09-12T02:37:39.221 INFO:teuthology.orchestra.run.smithi046.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-12T02:37:39.298 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 897 kB 00:00 2024-09-12T02:37:39.304 INFO:teuthology.orchestra.run.smithi046.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T02:37:39.426 INFO:teuthology.orchestra.run.smithi046.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.0 MB/s | 127 kB 00:00 2024-09-12T02:37:39.432 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.1 MB/s | 145 kB 00:00 2024-09-12T02:37:39.478 INFO:teuthology.orchestra.run.smithi145.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T02:37:39.589 INFO:teuthology.orchestra.run.smithi046.stdout:(17/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 17 MB 00:01 2024-09-12T02:37:39.630 INFO:teuthology.orchestra.run.smithi046.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 2.4 MB/s | 506 kB 00:00 2024-09-12T02:37:39.706 INFO:teuthology.orchestra.run.smithi046.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 602 kB/s | 45 kB 00:00 2024-09-12T02:37:39.814 INFO:teuthology.orchestra.run.smithi046.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.2 MB/s | 136 kB 00:00 2024-09-12T02:37:39.915 INFO:teuthology.orchestra.run.smithi046.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 162 kB 00:00 2024-09-12T02:37:40.023 INFO:teuthology.orchestra.run.smithi046.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.9 MB/s | 321 kB 00:00 2024-09-12T02:37:40.124 INFO:teuthology.orchestra.run.smithi046.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.9 MB/s | 301 kB 00:00 2024-09-12T02:37:40.215 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.0 MB/s | 2.4 MB 00:00 2024-09-12T02:37:40.224 INFO:teuthology.orchestra.run.smithi046.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 997 kB/s | 100 kB 00:00 2024-09-12T02:37:40.278 INFO:teuthology.orchestra.run.smithi145.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 1.4 MB/s | 2.4 MB 00:01 2024-09-12T02:37:40.299 INFO:teuthology.orchestra.run.smithi046.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 7.5 MB/s | 5.3 MB 00:00 2024-09-12T02:37:40.325 INFO:teuthology.orchestra.run.smithi046.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 850 kB/s | 85 kB 00:00 2024-09-12T02:37:40.425 INFO:teuthology.orchestra.run.smithi046.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-12T02:37:40.525 INFO:teuthology.orchestra.run.smithi046.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 318 kB/s | 32 kB 00:00 2024-09-12T02:37:40.674 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-12T02:37:40.750 INFO:teuthology.orchestra.run.smithi046.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 6.9 MB/s | 3.1 MB 00:00 2024-09-12T02:37:40.924 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-09-12T02:37:40.945 INFO:teuthology.orchestra.run.smithi145.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-12T02:37:41.145 INFO:teuthology.orchestra.run.smithi145.stdout:(9/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 6.1 MB/s | 21 MB 00:03 2024-09-12T02:37:41.170 INFO:teuthology.orchestra.run.smithi145.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 112 kB/s | 25 kB 00:00 2024-09-12T02:37:41.220 INFO:teuthology.orchestra.run.smithi145.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T02:37:41.271 INFO:teuthology.orchestra.run.smithi145.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 541 kB/s | 27 kB 00:00 2024-09-12T02:37:41.310 INFO:teuthology.orchestra.run.smithi046.stdout:(30/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 16 MB/s | 49 MB 00:03 2024-09-12T02:37:41.321 INFO:teuthology.orchestra.run.smithi145.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 633 kB/s | 32 kB 00:00 2024-09-12T02:37:41.324 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 7.6 MB/s | 21 MB 00:02 2024-09-12T02:37:41.334 INFO:teuthology.orchestra.run.smithi046.stdout:(31/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 181 kB/s | 146 kB 00:00 2024-09-12T02:37:41.390 INFO:teuthology.orchestra.run.smithi145.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-12T02:37:41.410 INFO:teuthology.orchestra.run.smithi046.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.4 MB/s | 3.6 MB 00:00 2024-09-12T02:37:41.416 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 4.7 MB 00:00 2024-09-12T02:37:41.435 INFO:teuthology.orchestra.run.smithi046.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T02:37:41.440 INFO:teuthology.orchestra.run.smithi145.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T02:37:41.517 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 252 kB/s | 25 kB 00:00 2024-09-12T02:37:41.524 INFO:teuthology.orchestra.run.smithi145.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 127 kB 00:00 2024-09-12T02:37:41.616 INFO:teuthology.orchestra.run.smithi145.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T02:37:41.744 INFO:teuthology.orchestra.run.smithi046.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 148 kB/s | 49 kB 00:00 2024-09-12T02:37:41.769 INFO:teuthology.orchestra.run.smithi046.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 49 kB/s | 16 kB 00:00 2024-09-12T02:37:41.911 INFO:teuthology.orchestra.run.smithi046.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 285 kB 00:00 2024-09-12T02:37:41.985 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 16 MB/s | 11 MB 00:00 2024-09-12T02:37:42.011 INFO:teuthology.orchestra.run.smithi046.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-12T02:37:42.034 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T02:37:42.036 INFO:teuthology.orchestra.run.smithi046.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 145 kB/s | 18 kB 00:00 2024-09-12T02:37:42.070 INFO:teuthology.orchestra.run.smithi046.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 2.5 MB/s | 765 kB 00:00 2024-09-12T02:37:42.075 INFO:teuthology.orchestra.run.smithi145.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-12T02:37:42.084 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 542 kB/s | 27 kB 00:00 2024-09-12T02:37:42.133 INFO:teuthology.orchestra.run.smithi145.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 776 kB/s | 45 kB 00:00 2024-09-12T02:37:42.134 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 635 kB/s | 32 kB 00:00 2024-09-12T02:37:42.162 INFO:teuthology.orchestra.run.smithi046.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-12T02:37:42.192 INFO:teuthology.orchestra.run.smithi145.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-12T02:37:42.201 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-12T02:37:42.221 INFO:teuthology.orchestra.run.smithi046.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-12T02:37:42.242 INFO:teuthology.orchestra.run.smithi145.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T02:37:42.246 INFO:teuthology.orchestra.run.smithi046.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-12T02:37:42.252 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T02:37:42.271 INFO:teuthology.orchestra.run.smithi046.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-12T02:37:42.293 INFO:teuthology.orchestra.run.smithi145.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T02:37:42.302 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T02:37:42.313 INFO:teuthology.orchestra.run.smithi046.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T02:37:42.343 INFO:teuthology.orchestra.run.smithi145.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T02:37:42.355 INFO:teuthology.orchestra.run.smithi046.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T02:37:42.361 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-12T02:37:42.381 INFO:teuthology.orchestra.run.smithi046.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-12T02:37:42.393 INFO:teuthology.orchestra.run.smithi145.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T02:37:42.406 INFO:teuthology.orchestra.run.smithi046.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T02:37:42.440 INFO:teuthology.orchestra.run.smithi046.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-12T02:37:42.460 INFO:teuthology.orchestra.run.smithi145.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.3 MB/s | 85 kB 00:00 2024-09-12T02:37:42.582 INFO:teuthology.orchestra.run.smithi046.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 756 kB/s | 107 kB 00:00 2024-09-12T02:37:42.607 INFO:teuthology.orchestra.run.smithi046.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-12T02:37:42.632 INFO:teuthology.orchestra.run.smithi046.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-12T02:37:42.636 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-12T02:37:42.657 INFO:teuthology.orchestra.run.smithi046.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-12T02:37:42.683 INFO:teuthology.orchestra.run.smithi046.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-12T02:37:42.686 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 908 kB/s | 45 kB 00:00 2024-09-12T02:37:42.708 INFO:teuthology.orchestra.run.smithi046.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-12T02:37:42.710 INFO:teuthology.orchestra.run.smithi145.stdout:(26/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-12T02:37:42.736 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-12T02:37:42.750 INFO:teuthology.orchestra.run.smithi046.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-09-12T02:37:42.761 INFO:teuthology.orchestra.run.smithi145.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-12T02:37:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-12T02:37:42.787 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T02:37:42.801 INFO:teuthology.orchestra.run.smithi046.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T02:37:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 639 kB/s | 32 kB 00:00 2024-09-12T02:37:42.826 INFO:teuthology.orchestra.run.smithi046.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-12T02:37:42.837 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T02:37:42.861 INFO:teuthology.orchestra.run.smithi145.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T02:37:42.885 INFO:teuthology.orchestra.run.smithi046.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-12T02:37:42.887 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T02:37:42.910 INFO:teuthology.orchestra.run.smithi046.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-12T02:37:42.935 INFO:teuthology.orchestra.run.smithi046.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-12T02:37:42.946 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T02:37:42.960 INFO:teuthology.orchestra.run.smithi046.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-12T02:37:42.994 INFO:teuthology.orchestra.run.smithi046.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-12T02:37:43.005 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-12T02:37:43.028 INFO:teuthology.orchestra.run.smithi046.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-12T02:37:43.053 INFO:teuthology.orchestra.run.smithi046.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-12T02:37:43.095 INFO:teuthology.orchestra.run.smithi046.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-12T02:37:43.120 INFO:teuthology.orchestra.run.smithi145.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 14 MB/s | 3.6 MB 00:00 2024-09-12T02:37:43.187 INFO:teuthology.orchestra.run.smithi046.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-12T02:37:43.189 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-12T02:37:43.229 INFO:teuthology.orchestra.run.smithi046.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-12T02:37:43.247 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T02:37:43.271 INFO:teuthology.orchestra.run.smithi046.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-12T02:37:43.331 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 382 kB/s | 32 kB 00:00 2024-09-12T02:37:43.338 INFO:teuthology.orchestra.run.smithi046.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-12T02:37:43.363 INFO:teuthology.orchestra.run.smithi046.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-12T02:37:43.389 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T02:37:43.405 INFO:teuthology.orchestra.run.smithi046.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T02:37:43.430 INFO:teuthology.orchestra.run.smithi046.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-12T02:37:43.481 INFO:teuthology.orchestra.run.smithi046.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T02:37:43.553 INFO:teuthology.orchestra.run.smithi145.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-12T02:37:43.564 INFO:teuthology.orchestra.run.smithi046.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-12T02:37:43.590 INFO:teuthology.orchestra.run.smithi046.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T02:37:43.590 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 18 MB/s | 3.6 MB 00:00 2024-09-12T02:37:43.603 INFO:teuthology.orchestra.run.smithi145.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:37:43.615 INFO:teuthology.orchestra.run.smithi046.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T02:37:43.640 INFO:teuthology.orchestra.run.smithi046.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-12T02:37:43.665 INFO:teuthology.orchestra.run.smithi046.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-12T02:37:43.670 INFO:teuthology.orchestra.run.smithi145.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 740 kB/s | 49 kB 00:00 2024-09-12T02:37:43.691 INFO:teuthology.orchestra.run.smithi046.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T02:37:43.716 INFO:teuthology.orchestra.run.smithi046.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-09-12T02:37:43.737 INFO:teuthology.orchestra.run.smithi145.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 246 kB/s | 16 kB 00:00 2024-09-12T02:37:43.800 INFO:teuthology.orchestra.run.smithi046.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-12T02:37:43.812 INFO:teuthology.orchestra.run.smithi145.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.7 MB/s | 285 kB 00:00 2024-09-12T02:37:43.833 INFO:teuthology.orchestra.run.smithi046.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-12T02:37:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T02:37:43.884 INFO:teuthology.orchestra.run.smithi046.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T02:37:43.905 INFO:teuthology.orchestra.run.smithi145.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-12T02:37:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-12T02:37:43.923 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 22 MB/s | 7.4 MB 00:00 2024-09-12T02:37:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-12T02:37:43.955 INFO:teuthology.orchestra.run.smithi145.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 364 kB/s | 18 kB 00:00 2024-09-12T02:37:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T02:37:43.974 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:37:43.985 INFO:teuthology.orchestra.run.smithi046.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T02:37:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T02:37:44.024 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 988 kB/s | 49 kB 00:00 2024-09-12T02:37:44.035 INFO:teuthology.orchestra.run.smithi046.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T02:37:44.074 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-12T02:37:44.089 INFO:teuthology.orchestra.run.smithi145.stdout:(38/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 2.8 MB/s | 11 MB 00:03 2024-09-12T02:37:44.125 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-12T02:37:44.144 INFO:teuthology.orchestra.run.smithi046.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T02:37:44.169 INFO:teuthology.orchestra.run.smithi046.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 311 kB/s | 42 kB 00:00 2024-09-12T02:37:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-12T02:37:44.195 INFO:teuthology.orchestra.run.smithi046.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-12T02:37:44.205 INFO:teuthology.orchestra.run.smithi145.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 319 kB/s | 80 kB 00:00 2024-09-12T02:37:44.220 INFO:teuthology.orchestra.run.smithi046.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T02:37:44.245 INFO:teuthology.orchestra.run.smithi046.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-12T02:37:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-12T02:37:44.297 INFO:teuthology.orchestra.run.smithi145.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-12T02:37:44.304 INFO:teuthology.orchestra.run.smithi046.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T02:37:44.330 INFO:teuthology.orchestra.run.smithi046.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-09-12T02:37:44.331 INFO:teuthology.orchestra.run.smithi145.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-12T02:37:44.355 INFO:teuthology.orchestra.run.smithi046.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T02:37:44.365 INFO:teuthology.orchestra.run.smithi145.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-12T02:37:44.381 INFO:teuthology.orchestra.run.smithi046.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T02:37:44.392 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 4.9 MB/s | 17 MB 00:03 2024-09-12T02:37:44.406 INFO:teuthology.orchestra.run.smithi046.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 887 kB/s | 45 kB 00:00 2024-09-12T02:37:44.415 INFO:teuthology.orchestra.run.smithi145.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-12T02:37:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 81 kB/s | 18 kB 00:00 2024-09-12T02:37:44.441 INFO:teuthology.orchestra.run.smithi145.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T02:37:44.466 INFO:teuthology.orchestra.run.smithi145.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T02:37:44.491 INFO:teuthology.orchestra.run.smithi145.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T02:37:44.516 INFO:teuthology.orchestra.run.smithi145.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T02:37:44.523 INFO:teuthology.orchestra.run.smithi046.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-09-12T02:37:44.568 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 533 kB/s | 80 kB 00:00 2024-09-12T02:37:44.618 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-12T02:37:44.668 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-12T02:37:44.682 INFO:teuthology.orchestra.run.smithi046.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-12T02:37:44.710 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-12T02:37:44.740 INFO:teuthology.orchestra.run.smithi046.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-09-12T02:37:44.750 INFO:teuthology.orchestra.run.smithi145.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-09-12T02:37:44.765 INFO:teuthology.orchestra.run.smithi046.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 791 kB/s | 19 kB 00:00 2024-09-12T02:37:44.790 INFO:teuthology.orchestra.run.smithi046.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-12T02:37:44.794 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-12T02:37:44.800 INFO:teuthology.orchestra.run.smithi145.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-12T02:37:44.816 INFO:teuthology.orchestra.run.smithi046.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:37:44.819 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-09-12T02:37:44.841 INFO:teuthology.orchestra.run.smithi046.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:37:44.842 INFO:teuthology.orchestra.run.smithi145.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-12T02:37:44.868 INFO:teuthology.orchestra.run.smithi145.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-12T02:37:44.878 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T02:37:44.883 INFO:teuthology.orchestra.run.smithi046.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-12T02:37:44.909 INFO:teuthology.orchestra.run.smithi046.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-12T02:37:44.918 INFO:teuthology.orchestra.run.smithi145.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-09-12T02:37:44.920 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-12T02:37:44.962 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T02:37:44.967 INFO:teuthology.orchestra.run.smithi046.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-12T02:37:44.968 INFO:teuthology.orchestra.run.smithi145.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-12T02:37:45.001 INFO:teuthology.orchestra.run.smithi046.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-12T02:37:45.018 INFO:teuthology.orchestra.run.smithi145.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T02:37:45.035 INFO:teuthology.orchestra.run.smithi046.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-12T02:37:45.060 INFO:teuthology.orchestra.run.smithi046.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:37:45.069 INFO:teuthology.orchestra.run.smithi145.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T02:37:45.085 INFO:teuthology.orchestra.run.smithi046.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-12T02:37:45.110 INFO:teuthology.orchestra.run.smithi046.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T02:37:45.152 INFO:teuthology.orchestra.run.smithi145.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-12T02:37:45.177 INFO:teuthology.orchestra.run.smithi046.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-12T02:37:45.203 INFO:teuthology.orchestra.run.smithi046.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-12T02:37:45.203 INFO:teuthology.orchestra.run.smithi145.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-12T02:37:45.228 INFO:teuthology.orchestra.run.smithi046.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-12T02:37:45.229 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T02:37:45.253 INFO:teuthology.orchestra.run.smithi046.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T02:37:45.279 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-12T02:37:45.287 INFO:teuthology.orchestra.run.smithi046.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-12T02:37:45.304 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-12T02:37:45.312 INFO:teuthology.orchestra.run.smithi046.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-12T02:37:45.337 INFO:teuthology.orchestra.run.smithi046.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-12T02:37:45.338 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 369 kB/s | 22 kB 00:00 2024-09-12T02:37:45.363 INFO:teuthology.orchestra.run.smithi046.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-12T02:37:45.380 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-12T02:37:45.388 INFO:teuthology.orchestra.run.smithi046.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-12T02:37:45.405 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-12T02:37:45.413 INFO:teuthology.orchestra.run.smithi046.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-12T02:37:45.439 INFO:teuthology.orchestra.run.smithi046.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T02:37:45.464 INFO:teuthology.orchestra.run.smithi046.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:37:45.498 INFO:teuthology.orchestra.run.smithi046.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-12T02:37:45.531 INFO:teuthology.orchestra.run.smithi046.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-12T02:37:45.565 INFO:teuthology.orchestra.run.smithi046.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-09-12T02:37:45.590 INFO:teuthology.orchestra.run.smithi046.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-12T02:37:45.616 INFO:teuthology.orchestra.run.smithi046.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T02:37:45.641 INFO:teuthology.orchestra.run.smithi046.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T02:37:45.666 INFO:teuthology.orchestra.run.smithi046.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-12T02:37:45.691 INFO:teuthology.orchestra.run.smithi046.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-12T02:37:45.711 INFO:teuthology.orchestra.run.smithi145.stdout:(58/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 11 MB/s | 49 MB 00:04 2024-09-12T02:37:45.734 INFO:teuthology.orchestra.run.smithi046.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-12T02:37:45.736 INFO:teuthology.orchestra.run.smithi145.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.2 MB/s | 662 kB 00:00 2024-09-12T02:37:45.759 INFO:teuthology.orchestra.run.smithi046.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 881 kB/s | 59 kB 00:00 2024-09-12T02:37:45.786 INFO:teuthology.orchestra.run.smithi046.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 692 kB/s | 36 kB 00:00 2024-09-12T02:37:45.787 INFO:teuthology.orchestra.run.smithi145.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-12T02:37:45.811 INFO:teuthology.orchestra.run.smithi046.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-12T02:37:45.822 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 11 MB/s | 49 MB 00:04 2024-09-12T02:37:45.836 INFO:teuthology.orchestra.run.smithi046.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 968 kB/s | 74 kB 00:00 2024-09-12T02:37:45.837 INFO:teuthology.orchestra.run.smithi145.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-12T02:37:45.847 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 41 kB/s | 18 kB 00:00 2024-09-12T02:37:45.861 INFO:teuthology.orchestra.run.smithi046.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T02:37:45.887 INFO:teuthology.orchestra.run.smithi046.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T02:37:45.906 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T02:37:45.912 INFO:teuthology.orchestra.run.smithi046.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-12T02:37:45.929 INFO:teuthology.orchestra.run.smithi145.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-12T02:37:45.932 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-12T02:37:45.937 INFO:teuthology.orchestra.run.smithi046.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:37:45.954 INFO:teuthology.orchestra.run.smithi145.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 2.3 MB/s | 581 kB 00:00 2024-09-12T02:37:45.971 INFO:teuthology.orchestra.run.smithi046.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T02:37:45.982 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-09-12T02:37:46.007 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-12T02:37:46.013 INFO:teuthology.orchestra.run.smithi145.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-12T02:37:46.030 INFO:teuthology.orchestra.run.smithi046.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T02:37:46.038 INFO:teuthology.orchestra.run.smithi145.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T02:37:46.063 INFO:teuthology.orchestra.run.smithi145.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-12T02:37:46.074 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-12T02:37:46.099 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 266 kB/s | 24 kB 00:00 2024-09-12T02:37:46.141 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-12T02:37:46.175 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-12T02:37:46.200 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-09-12T02:37:46.226 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T02:37:46.230 INFO:teuthology.orchestra.run.smithi046.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T02:37:46.251 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-12T02:37:46.272 INFO:teuthology.orchestra.run.smithi046.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 9.9 MB/s | 2.3 MB 00:00 2024-09-12T02:37:46.339 INFO:teuthology.orchestra.run.smithi046.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-12T02:37:46.359 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T02:37:46.364 INFO:teuthology.orchestra.run.smithi046.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 445 kB/s | 41 kB 00:00 2024-09-12T02:37:46.389 INFO:teuthology.orchestra.run.smithi046.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-12T02:37:46.448 INFO:teuthology.orchestra.run.smithi046.stdout:(154/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.6 MB/s | 3.2 MB 00:00 2024-09-12T02:37:46.451 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-12T02:37:46.472 INFO:teuthology.orchestra.run.smithi145.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-09-12T02:37:46.490 INFO:teuthology.orchestra.run.smithi046.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-12T02:37:46.518 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-12T02:37:46.540 INFO:teuthology.orchestra.run.smithi046.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-12T02:37:46.544 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-12T02:37:46.546 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:37:46.547 INFO:teuthology.orchestra.run.smithi046.stdout:Total 24 MB/s | 226 MB 00:09 2024-09-12T02:37:46.577 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-12T02:37:46.611 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T02:37:46.636 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T02:37:46.672 INFO:teuthology.orchestra.run.smithi145.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-12T02:37:46.695 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T02:37:46.731 INFO:teuthology.orchestra.run.smithi145.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T02:37:46.779 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T02:37:46.804 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T02:37:46.879 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-12T02:37:46.890 INFO:teuthology.orchestra.run.smithi145.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-12T02:37:46.904 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-12T02:37:46.938 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-12T02:37:46.963 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 228 kB/s | 19 kB 00:00 2024-09-12T02:37:46.997 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-12T02:37:47.022 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 415 kB/s | 35 kB 00:00 2024-09-12T02:37:47.056 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T02:37:47.081 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T02:37:47.098 INFO:teuthology.orchestra.run.smithi145.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-12T02:37:47.132 INFO:teuthology.orchestra.run.smithi145.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-09-12T02:37:47.148 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-09-12T02:37:47.157 INFO:teuthology.orchestra.run.smithi145.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T02:37:47.173 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-12T02:37:47.198 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 77 kB 00:00 2024-09-12T02:37:47.207 INFO:teuthology.orchestra.run.smithi145.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T02:37:47.232 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-12T02:37:47.258 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-12T02:37:47.291 INFO:teuthology.orchestra.run.smithi145.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-12T02:37:47.291 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-12T02:37:47.317 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-12T02:37:47.342 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 54 kB 00:00 2024-09-12T02:37:47.375 INFO:teuthology.orchestra.run.smithi145.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-12T02:37:47.400 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 764 kB/s | 44 kB 00:00 2024-09-12T02:37:47.426 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): python3-toml-0.10.2-6.el9.noarch.rpm 500 kB/s | 42 kB 00:00 2024-09-12T02:37:47.433 INFO:teuthology.orchestra.run.smithi145.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 442 kB/s | 26 kB 00:00 2024-09-12T02:37:47.459 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:37:47.485 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-12T02:37:47.492 INFO:teuthology.orchestra.run.smithi145.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-12T02:37:47.518 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-12T02:37:47.550 INFO:teuthology.orchestra.run.smithi145.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T02:37:47.552 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-12T02:37:47.577 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-12T02:37:47.609 INFO:teuthology.orchestra.run.smithi145.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-12T02:37:47.627 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T02:37:47.652 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-09-12T02:37:47.686 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-09-12T02:37:47.711 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-09-12T02:37:47.793 INFO:teuthology.orchestra.run.smithi145.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.4 MB/s | 5.8 MB 00:01 2024-09-12T02:37:47.813 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-12T02:37:47.862 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-12T02:37:47.868 INFO:teuthology.orchestra.run.smithi145.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 8.1 MB/s | 6.0 MB 00:00 2024-09-12T02:37:47.895 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-12T02:37:47.918 INFO:teuthology.orchestra.run.smithi145.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-12T02:37:47.944 INFO:teuthology.orchestra.run.smithi145.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-12T02:37:47.962 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 375 kB/s | 19 kB 00:00 2024-09-12T02:37:47.969 INFO:teuthology.orchestra.run.smithi145.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T02:37:48.059 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-12T02:37:48.059 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-12T02:37:48.059 INFO:teuthology.orchestra.run.smithi145.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-12T02:37:48.059 INFO:teuthology.orchestra.run.smithi145.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-12T02:37:48.060 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-09-12T02:37:48.060 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-09-12T02:37:48.060 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 514 kB/s | 26 kB 00:00 2024-09-12T02:37:48.060 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-12T02:37:48.086 INFO:teuthology.orchestra.run.smithi145.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-12T02:37:48.089 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-12T02:37:48.112 INFO:teuthology.orchestra.run.smithi145.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-12T02:37:48.114 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-12T02:37:48.137 INFO:teuthology.orchestra.run.smithi145.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T02:37:48.162 INFO:teuthology.orchestra.run.smithi145.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T02:37:48.164 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-12T02:37:48.190 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:37:48.213 INFO:teuthology.orchestra.run.smithi145.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-12T02:37:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-12T02:37:48.240 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-12T02:37:48.263 INFO:teuthology.orchestra.run.smithi145.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-12T02:37:48.265 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-12T02:37:48.291 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-12T02:37:48.321 INFO:teuthology.orchestra.run.smithi145.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:37:48.324 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T02:37:48.350 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-12T02:37:48.371 INFO:teuthology.orchestra.run.smithi145.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-12T02:37:48.375 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T02:37:48.409 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T02:37:48.413 INFO:teuthology.orchestra.run.smithi145.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-12T02:37:48.434 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-12T02:37:48.459 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-12T02:37:48.484 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-12T02:37:48.489 INFO:teuthology.orchestra.run.smithi145.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 5.1 MB 00:00 2024-09-12T02:37:48.510 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-12T02:37:48.522 INFO:teuthology.orchestra.run.smithi145.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-12T02:37:48.539 INFO:teuthology.orchestra.run.smithi145.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T02:37:48.577 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-09-12T02:37:48.590 INFO:teuthology.orchestra.run.smithi145.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-12T02:37:48.602 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-09-12T02:37:48.615 INFO:teuthology.orchestra.run.smithi145.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-09-12T02:37:48.627 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:37:48.640 INFO:teuthology.orchestra.run.smithi145.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-12T02:37:48.653 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:37:48.666 INFO:teuthology.orchestra.run.smithi145.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-12T02:37:48.678 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:37:48.708 INFO:teuthology.orchestra.run.smithi145.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-12T02:37:48.711 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T02:37:48.737 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T02:37:48.749 INFO:teuthology.orchestra.run.smithi145.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T02:37:48.775 INFO:teuthology.orchestra.run.smithi145.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 844 kB/s | 19 kB 00:00 2024-09-12T02:37:48.779 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-12T02:37:48.808 INFO:teuthology.orchestra.run.smithi145.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-09-12T02:37:48.812 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T02:37:48.834 INFO:teuthology.orchestra.run.smithi145.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:37:48.838 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-09-12T02:37:48.859 INFO:teuthology.orchestra.run.smithi145.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:37:48.863 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-12T02:37:48.884 INFO:teuthology.orchestra.run.smithi145.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T02:37:48.897 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-09-12T02:37:48.909 INFO:teuthology.orchestra.run.smithi145.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T02:37:48.922 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-12T02:37:48.941 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 816 kB/s | 36 kB 00:00 2024-09-12T02:37:48.960 INFO:teuthology.orchestra.run.smithi145.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-12T02:37:48.966 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-12T02:37:48.992 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-12T02:37:48.993 INFO:teuthology.orchestra.run.smithi145.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T02:37:49.017 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T02:37:49.019 INFO:teuthology.orchestra.run.smithi145.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:37:49.045 INFO:teuthology.orchestra.run.smithi145.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:37:49.050 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-12T02:37:49.069 INFO:teuthology.orchestra.run.smithi145.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-12T02:37:49.076 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-12T02:37:49.095 INFO:teuthology.orchestra.run.smithi145.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-12T02:37:49.101 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:37:49.126 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T02:37:49.128 INFO:teuthology.orchestra.run.smithi145.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T02:37:49.162 INFO:teuthology.orchestra.run.smithi145.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T02:37:49.176 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:37:49.229 INFO:teuthology.orchestra.run.smithi145.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-09-12T02:37:49.254 INFO:teuthology.orchestra.run.smithi145.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-12T02:37:49.288 INFO:teuthology.orchestra.run.smithi145.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T02:37:49.302 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-09-12T02:37:49.313 INFO:teuthology.orchestra.run.smithi145.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T02:37:49.338 INFO:teuthology.orchestra.run.smithi145.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-12T02:37:49.363 INFO:teuthology.orchestra.run.smithi145.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 943 kB/s | 23 kB 00:00 2024-09-12T02:37:49.389 INFO:teuthology.orchestra.run.smithi145.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-12T02:37:49.414 INFO:teuthology.orchestra.run.smithi145.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-12T02:37:49.419 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-09-12T02:37:49.469 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-12T02:37:49.473 INFO:teuthology.orchestra.run.smithi145.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-12T02:37:49.494 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 541 kB/s | 41 kB 00:00 2024-09-12T02:37:49.507 INFO:teuthology.orchestra.run.smithi145.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-09-12T02:37:49.540 INFO:teuthology.orchestra.run.smithi145.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-12T02:37:49.553 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-12T02:37:49.574 INFO:teuthology.orchestra.run.smithi145.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-12T02:37:49.595 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-12T02:37:49.599 INFO:teuthology.orchestra.run.smithi145.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-12T02:37:49.624 INFO:teuthology.orchestra.run.smithi145.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-12T02:37:49.650 INFO:teuthology.orchestra.run.smithi145.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T02:37:49.678 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-12T02:37:49.683 INFO:teuthology.orchestra.run.smithi145.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-12T02:37:49.700 INFO:teuthology.orchestra.run.smithi145.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-12T02:37:49.717 INFO:teuthology.orchestra.run.smithi145.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 921 kB/s | 31 kB 00:00 2024-09-12T02:37:49.728 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 5.2 MB/s | 3.2 MB 00:00 2024-09-12T02:37:49.734 INFO:teuthology.orchestra.run.smithi145.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-12T02:37:49.735 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:37:49.735 INFO:teuthology.orchestra.run.smithi196.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T02:37:49.776 INFO:teuthology.orchestra.run.smithi145.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:37:49.793 INFO:teuthology.orchestra.run.smithi145.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-12T02:37:49.818 INFO:teuthology.orchestra.run.smithi145.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-12T02:37:49.844 INFO:teuthology.orchestra.run.smithi145.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:37:49.869 INFO:teuthology.orchestra.run.smithi145.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T02:37:49.894 INFO:teuthology.orchestra.run.smithi145.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-12T02:37:49.920 INFO:teuthology.orchestra.run.smithi145.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-12T02:37:49.945 INFO:teuthology.orchestra.run.smithi145.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T02:37:50.112 INFO:teuthology.orchestra.run.smithi145.stdout:(147/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-09-12T02:37:50.154 INFO:teuthology.orchestra.run.smithi145.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-12T02:37:50.271 INFO:teuthology.orchestra.run.smithi145.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-12T02:37:50.298 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-12T02:37:50.298 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-12T02:37:50.346 INFO:teuthology.orchestra.run.smithi145.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-12T02:37:50.541 INFO:teuthology.orchestra.run.smithi145.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 209 kB/s | 41 kB 00:00 2024-09-12T02:37:50.613 INFO:teuthology.orchestra.run.smithi145.stdout:(152/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.3 MB/s | 3.2 MB 00:00 2024-09-12T02:37:50.638 INFO:teuthology.orchestra.run.smithi145.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.5 MB/s | 249 kB 00:00 2024-09-12T02:37:50.680 INFO:teuthology.orchestra.run.smithi145.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-12T02:37:50.730 INFO:teuthology.orchestra.run.smithi145.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-09-12T02:37:50.822 INFO:teuthology.orchestra.run.smithi145.stdout:(156/156): librados2-19.1.1-298.gb53b6d3e.el9.x 3.9 MB/s | 3.4 MB 00:00 2024-09-12T02:37:50.827 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:37:50.828 INFO:teuthology.orchestra.run.smithi145.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-12T02:37:50.918 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-12T02:37:51.080 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-12T02:37:51.080 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-12T02:37:52.007 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-12T02:37:52.159 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-12T02:37:52.159 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-12T02:37:53.341 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-12T02:37:53.341 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-12T02:37:54.405 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-12T02:37:54.406 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-12T02:37:57.034 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-12T02:37:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:37:57.184 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:37:57.247 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:37:57.287 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:37:57.312 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:37:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:37:57.723 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:37:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:37:57.955 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:37:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:37:58.083 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:37:58.111 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:37:58.283 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:37:58.350 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:37:58.377 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:37:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:37:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:37:58.575 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:37:58.604 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:37:58.628 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:37:58.825 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:37:58.995 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:37:59.132 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:37:59.243 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:37:59.321 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:37:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:37:59.565 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-12T02:37:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:37:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:37:59.770 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:37:59.810 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:37:59.845 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:37:59.882 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:37:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:37:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:38:00.028 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:38:00.035 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:38:00.072 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:38:00.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:38:00.104 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:38:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:38:00.132 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:38:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:38:00.203 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:38:00.273 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:38:00.302 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:38:00.345 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:38:00.425 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:38:00.464 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:38:00.500 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:38:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:38:00.536 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:38:00.609 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:00.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:38:00.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:38:00.671 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-12T02:38:00.723 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:38:00.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:00.809 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:38:00.836 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:00.856 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:38:00.865 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:38:00.893 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:38:00.924 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:38:00.955 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:38:00.960 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:38:00.985 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:38:01.040 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:38:01.041 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:38:01.095 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:38:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:01.227 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:01.249 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:38:01.313 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:38:01.342 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:38:01.366 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:38:01.368 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:38:01.514 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:38:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:38:01.578 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:01.603 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:38:01.645 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:38:01.688 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:38:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:38:01.748 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:38:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:38:01.899 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:38:01.926 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:01.944 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:38:01.955 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:01.996 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:38:02.014 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:02.047 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:02.054 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:38:02.076 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:38:02.092 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:02.127 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:38:02.172 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:38:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:38:02.245 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:38:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:38:02.298 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:38:02.299 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:38:02.318 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:02.333 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:02.343 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:38:02.385 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:38:02.425 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:38:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:38:02.518 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:38:02.522 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:38:02.531 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:38:02.552 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:38:02.556 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:38:02.580 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:38:02.590 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:38:02.655 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:38:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:38:02.757 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:38:02.770 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:38:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:38:02.812 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:38:02.847 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:38:02.850 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:38:02.877 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:38:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:38:02.960 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:38:02.976 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:38:02.999 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:38:03.000 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:38:03.032 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:38:03.040 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:38:03.072 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:38:03.074 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:38:03.097 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:38:03.100 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:38:03.149 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:38:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:38:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:38:03.224 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:38:03.265 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:38:03.319 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:03.323 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:38:03.334 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:38:03.363 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:38:03.380 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:38:03.406 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:38:03.454 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:38:03.457 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:38:03.498 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:38:03.538 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:38:03.543 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:38:03.568 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:38:03.587 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:38:03.602 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:38:03.635 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:38:03.640 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:38:03.654 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:38:03.677 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:38:03.711 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:38:03.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:38:03.783 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:38:03.801 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:38:03.817 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:38:03.859 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:38:03.882 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:38:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:38:03.926 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:38:03.958 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:38:03.999 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:38:04.002 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:38:04.034 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:38:04.070 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:38:04.148 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:38:04.161 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:38:04.167 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:38:04.209 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:38:04.241 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:38:04.244 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:38:04.273 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:38:04.284 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:38:04.316 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:38:04.361 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:38:04.363 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:38:04.394 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:38:04.414 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:38:04.437 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:38:04.466 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:38:04.536 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:38:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:38:04.589 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:38:04.633 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:38:04.664 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:38:04.687 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:38:04.761 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:38:04.792 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:38:04.805 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:38:04.828 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:38:04.858 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:38:04.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:38:04.919 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:38:04.933 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:38:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:38:04.981 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:38:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:38:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:38:05.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:38:05.139 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:38:05.171 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:38:05.190 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:38:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:38:05.231 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:38:05.309 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:38:05.334 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:38:05.342 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:38:05.385 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:38:05.470 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:38:05.499 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:38:05.535 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:38:05.550 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:38:05.557 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:38:05.573 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:38:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:38:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:38:05.646 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:38:05.661 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:38:05.693 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:38:05.717 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:38:05.786 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:38:05.820 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:38:05.842 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:38:05.896 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:38:05.946 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:38:05.969 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:38:05.982 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:38:06.001 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:38:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:38:06.037 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:38:06.051 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:38:06.084 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:38:06.092 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:38:06.124 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:38:06.145 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:38:06.154 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:38:06.188 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:38:06.196 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:38:06.222 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:38:06.238 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:38:06.272 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:38:06.306 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:38:06.355 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:38:06.374 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:38:06.409 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:38:06.415 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:38:06.456 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:38:06.461 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:38:06.476 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:38:06.493 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:38:06.559 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:38:06.590 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:38:06.595 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:38:06.631 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:38:06.644 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:38:06.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:38:06.697 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:38:06.876 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:38:07.115 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:38:07.154 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:38:07.194 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:38:07.230 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:38:07.267 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:38:07.299 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:38:07.337 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:38:07.371 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:38:07.372 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:38:07.405 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:38:07.418 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:38:07.446 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:38:07.451 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:38:07.481 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:38:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:38:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:38:07.598 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:38:07.611 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:38:07.642 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:38:07.643 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:38:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:38:07.753 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:38:07.781 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:38:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:38:07.822 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:38:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:38:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:38:07.947 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:38:07.964 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:38:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:38:08.074 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:38:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:38:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:08.402 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:38:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:38:08.548 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:38:08.599 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:38:08.632 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:38:08.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:38:08.732 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:38:08.787 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:38:08.833 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:38:08.852 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:38:08.921 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:38:08.951 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:38:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:38:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:09.076 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:09.106 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:38:09.133 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:38:09.166 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:38:09.252 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:38:09.904 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:38:10.013 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:38:10.109 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:38:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:38:10.219 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:38:10.253 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:38:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:38:10.824 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:11.074 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:11.104 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:38:11.148 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:38:11.247 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:38:11.328 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:38:11.357 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:38:11.389 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:38:11.627 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:38:11.660 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:11.701 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:38:11.715 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:11.719 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:11.744 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:11.745 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:38:11.835 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:38:11.865 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:38:11.910 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:38:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:12.388 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:38:12.421 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:12.449 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:38:12.449 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:38:12.449 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:12.491 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:12.492 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:38:12.515 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:38:14.070 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:14.109 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:14.674 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:14.712 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:14.806 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:14.806 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:38:14.806 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:38:14.806 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:14.836 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:14.847 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:14.879 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:15.447 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:38:15.490 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:15.582 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:38:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:38:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:38:15.582 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:15.624 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:24.638 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:24.638 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-09-12T02:38:24.639 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:24.830 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:24.847 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:24.848 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:38:24.848 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:26.411 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:26.411 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:38:26.411 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:26.412 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-12T02:38:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:26.701 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:26.716 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:26.716 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:26.716 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:38:26.716 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:27.274 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:38:27.340 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:38:27.404 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-09-12T02:38:27.536 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:27.601 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:38:27.653 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:38:27.712 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:27.727 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:38:27.728 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:27.728 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:38:27.728 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:27.750 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:38:27.850 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:38:28.009 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:38:28.062 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:38:28.093 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:38:28.120 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:38:28.178 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:28.204 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:28.205 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:28.205 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:38:28.205 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:28.205 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:28.205 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:28.429 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:28.452 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:28.572 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:28.603 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:28.634 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:38:28.699 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:38:28.726 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:28.911 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:28.911 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:28.911 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:38:28.911 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:28.912 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:28.912 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:29.018 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:38:29.083 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:38:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:38:29.231 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:38:29.257 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:29.257 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:38:29.258 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:29.258 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:38:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:29.342 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:38:29.406 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:38:29.482 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:29.484 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:38:29.505 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:29.581 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:38:29.723 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:38:29.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:38:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:38:29.836 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:29.849 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:38:29.869 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:29.941 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:29.964 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:38:29.964 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:29.964 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:38:29.964 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:29.965 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:38:29.965 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:30.071 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:38:30.128 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:38:30.184 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:38:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:30.346 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:38:30.371 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:38:30.409 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:38:30.425 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:38:30.437 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:30.503 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:38:30.600 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:38:30.740 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:38:30.784 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:38:30.815 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:38:30.843 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:38:30.887 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:30.911 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:38:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:31.268 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:31.300 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:38:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:31.331 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:38:31.396 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:38:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:38:31.478 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:38:31.514 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:32.465 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:38:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:38:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:32.500 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:32.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:32.567 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:32.592 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:32.592 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:38:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:32.618 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:32.776 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:38:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:32.868 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:32.879 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:32.985 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:32.987 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:38:33.010 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:33.011 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:33.011 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:38:33.011 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:33.011 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:33.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:33.044 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:38:33.461 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:38:33.499 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:33.603 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:38:33.771 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:38:33.802 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:38:33.825 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:33.869 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:33.880 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:38:33.962 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:38:33.979 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:38:34.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:34.032 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:34.044 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:34.088 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:34.110 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:34.111 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:34.111 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:38:34.111 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:34.155 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:34.178 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:34.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:34.512 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:34.964 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:38:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:35.020 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:38:35.021 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:35.021 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:38:35.021 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:35.021 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:38:35.021 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:35.064 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:35.086 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:38:35.087 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:35.087 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:38:35.087 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:35.148 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:35.170 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:38:35.171 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:35.171 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:38:35.171 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:35.171 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:38:35.171 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:35.473 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:38:35.497 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:37.848 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:38:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:38:37.930 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:38:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:38:37.989 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:38:38.192 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:38:38.244 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:38:38.266 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:38:38.295 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:38:38.341 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:38:38.930 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:38.973 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:38.994 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:39.059 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:38:39.109 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:38:39.150 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:38:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:38:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:38:39.402 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:38:39.448 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:38:39.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:38:39.508 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:38:39.562 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:38:39.891 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:38:39.978 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:38:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:38:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:38:40.122 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:38:40.135 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:38:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:38:40.161 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:40.176 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:38:40.181 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:38:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:38:40.207 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:40.208 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:38:40.219 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:40.235 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:38:40.239 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:40.291 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:40.292 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:38:40.407 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:40.407 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:40.433 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:38:40.482 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:38:40.512 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:38:40.550 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:38:40.588 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:38:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:40.687 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:40.755 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:41.062 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:38:41.165 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:41.165 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:38:41.211 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:41.212 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:38:41.231 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:38:41.304 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:38:41.335 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:38:41.361 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:38:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:38:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:41.588 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:41.613 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:38:41.619 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:41.619 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:41.798 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:41.798 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:41.882 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:41.947 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:42.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:38:42.140 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:38:42.189 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:38:42.293 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:38:42.326 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:38:42.352 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:38:42.395 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:38:42.395 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:42.465 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:38:42.465 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:42.563 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:38:42.588 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:38:42.593 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:42.593 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:42.773 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:38:42.773 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:42.857 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:38:42.931 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:38:46.458 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:38:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:38:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:38:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:38:46.460 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:38:46.461 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:38:46.462 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:38:46.463 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:38:46.464 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:38:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:38:46.466 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:38:46.467 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:38:46.468 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:38:46.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:38:46.470 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:38:46.471 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:38:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:38:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:38:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:38:46.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:38:47.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:38:47.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:38:47.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:38:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:38:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:38:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:38:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:38:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:38:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:38:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:38:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:38:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:38:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:38:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:38:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:38:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:38:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:38:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:38:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:38:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:38:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:38:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:38:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:38:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:38:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:38:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:38:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:38:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:38:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:49.339 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.340 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.341 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:38:49.342 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:38:49.343 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:38:49.344 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:38:49.345 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:38:49.346 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:38:49.347 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:38:49.348 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:38:49.349 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:38:49.350 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:38:49.351 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:38:49.351 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:38:49.351 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-12T02:38:49.898 DEBUG:teuthology.parallel:result is None 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:50.541 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.543 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:38:50.544 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:38:50.545 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:38:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:38:50.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:38:50.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:38:50.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:38:50.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:38:50.551 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:50.552 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:38:50.553 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-12T02:38:50.936 DEBUG:teuthology.parallel:result is None 2024-09-12T02:38:51.535 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:51.536 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.537 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.538 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:38:51.540 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:38:51.541 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:38:51.542 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:38:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:38:51.544 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:38:51.545 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:38:51.546 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:38:51.547 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:38:51.548 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:38:51.548 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:38:51.548 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-12T02:38:51.949 DEBUG:teuthology.parallel:result is None 2024-09-12T02:38:51.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:52.096 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:38:52.123 INFO:teuthology.orchestra.run.smithi046.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.124 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.124 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:38:52.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:52.259 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:38:52.283 INFO:teuthology.orchestra.run.smithi145.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.284 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.284 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:38:52.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:52.420 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:38:52.447 INFO:teuthology.orchestra.run.smithi196.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.447 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:38:52.447 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:38:52.449 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T02:38:52.449 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:38:52.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:38:52.483 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:38:52.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:38:52.514 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:38:52.514 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:38:52.546 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T02:38:52.547 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:38:52.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:38:52.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:38:52.648 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:38:52.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:38:52.678 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:38:52.748 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:38:52.748 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:38:52.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:38:52.846 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T02:38:52.846 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:38:52.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:38:52.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:38:52.947 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:38:52.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:38:52.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:38:53.047 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:38:53.047 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:38:53.077 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:38:53.144 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T02:38:53.145 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:38:53.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:38:53.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:38:53.244 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:38:53.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:38:53.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:38:53.342 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:38:53.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:38:53.372 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:38:53.441 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T02:38:53.542 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-12T02:38:53.542 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:53.543 INFO:tasks.cephadm:Cluster fsid is 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:38:53.543 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T02:38:53.543 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.145', 'mon.c': '172.21.15.196'} 2024-09-12T02:38:53.543 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-09-12T02:38:53.543 INFO:tasks.cephadm:First mgr is x 2024-09-12T02:38:53.543 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T02:38:53.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-09-12T02:38:53.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-09-12T02:38:53.604 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-12T02:38:53.635 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T02:38:53.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:53.773 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T02:38:53.911 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:53.912 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:38:53.912 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:38:53.912 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:54.293 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:38 /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:54.293 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:54.672 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:38 /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:54.672 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:55.053 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:38 /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:55.054 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:55.075 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:55.095 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:38:55.129 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T02:38:55.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:38:55.131 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:38:55.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:38:56.764 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:38:56.769 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:38:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:39:23.339 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-12T02:39:23.340 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T02:39:24.118 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:39:29.162 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-12T02:39:29.163 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-12T02:39:29.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-09-12T02:39:29.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-09-12T02:39:29.252 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-12T02:39:29.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-09-12T02:39:29.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-09-12T02:39:29.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-12T02:39:29.370 INFO:tasks.cephadm:Writing seed config... 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T02:39:29.371 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T02:39:29.372 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T02:39:29.373 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T02:39:29.374 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-12T02:39:29.375 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:39:29.375 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T02:39:29.394 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2691fc0c-70b0-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T02:39:29.395 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T02:39:29.437 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T02:39:29.480 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T02:39:29.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:39:29.697 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:39:29.697 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '2691fc0c-70b0-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-09-12T02:39:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T02:39:29.698 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-09-12T02:39:29.723 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:39:29.723 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-09-12T02:39:29.723 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-09-12T02:39:29.731 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:39:29.731 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:39:29.738 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:39:29.738 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:39:29.744 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-12T02:39:29.751 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-12T02:39:29.751 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:39:29.751 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-09-12T02:39:29.774 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:39:29.775 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T02:39:29.775 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-09-12T02:39:29.775 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-09-12T02:39:29.781 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:39:29.782 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:39:29.788 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:39:29.788 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:39:29.795 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-12T02:39:29.801 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 140439810798592 on /run/cephadm/2691fc0c-70b0-11ef-bcea-c7b262605968.lock 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 140439810798592 acquired on /run/cephadm/2691fc0c-70b0-11ef-bcea-c7b262605968.lock 2024-09-12T02:39:29.802 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-09-12T02:39:29.803 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-09-12T02:39:29.803 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-12T02:39:29.806 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-09-12T02:39:29.806 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-09-12T02:39:29.808 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T02:39:29.809 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T02:39:29.809 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1780sec hoplimit 64 pref medium 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-12T02:39:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:39:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-12T02:39:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-12T02:39:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T02:39:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T02:39:29.813 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T02:39:33.171 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:39:33.172 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T02:39:39.576 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:39:39.576 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:39:39.577 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T02:39:40.631 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-09-12T02:39:40.632 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-09-12T02:39:41.620 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBsVOJm5+qTOxAAqaNPhbXUo/h/PwSsMv7/yw== 2024-09-12T02:39:42.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBtVOJmdptdORAAja7bqVOZuDD2DJo2XhzRxQ== 2024-09-12T02:39:43.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQBuVOJmv4/KLxAAWo9KuwAozA0GULVnLkwQsA== 2024-09-12T02:39:43.365 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:39:44.271 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:39:44.272 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-09-12T02:39:47.769 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-09-12T02:39:48.081 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T02:39:48.227 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T02:39:48.404 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target. 2024-09-12T02:39:48.405 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target. 2024-09-12T02:39:48.638 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:39:48.638 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T02:39:48.813 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target.wants/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@.service. 2024-09-12T02:39:49.350 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:39:49.350 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-12T02:39:49.357 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:39:49.357 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:39:49.357 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-12T02:39:49.358 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:39:49.358 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-09-12T02:39:49.358 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-09-12T02:39:50.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:50 smithi046 ceph-mon[30583]: from='client.? 172.21.15.46:0/1365272598' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T02:39:50.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.672946s) 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T02:39:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.903+0000 7f7ae6ceb640 1 Processor -- start 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.903+0000 7f7ae6ceb640 1 -- start start 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.903+0000 7f7ae6ceb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.903+0000 7f7ae6ceb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae0106e50 con 0x7f7ae0106480 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.904+0000 7f7ae4a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.904+0000 7f7ae4a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40934/0 (socket says 172.21.15.46:40934) 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.904+0000 7f7ae4a87640 1 -- 172.21.15.46:0/1418426693 learned_addr learned my addr 172.21.15.46:0/1418426693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.904+0000 7f7ae4a87640 1 -- 172.21.15.46:0/1418426693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae0107680 con 0x7f7ae0106480 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7ae4a87640 1 --2- 172.21.15.46:0/1418426693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009820 tx=0x7f7ad402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17cb873e07414e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:50.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7adf7fe640 1 -- 172.21.15.46:0/1418426693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ad4037070 con 0x7f7ae0106480 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7adf7fe640 1 -- 172.21.15.46:0/1418426693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4032440 con 0x7f7ae0106480 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7adf7fe640 1 -- 172.21.15.46:0/1418426693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ad4032740 con 0x7f7ae0106480 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1418426693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 msgr2=0x7f7ae0106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.906+0000 7f7ae6ceb640 1 --2- 172.21.15.46:0/1418426693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009820 tx=0x7f7ad402a270 comp rx=0 tx=0).stop 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1418426693 shutdown_connections 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 --2- 172.21.15.46:0/1418426693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae0106880 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1418426693 >> 172.21.15.46:0/1418426693 conn(0x7f7ae0101bf0 msgr2=0x7f7ae0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1418426693 shutdown_connections 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1418426693 wait complete. 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 Processor -- start 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- start start 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:50.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.907+0000 7f7ae6ceb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae0107c20 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae4a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae4a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40942/0 (socket says 172.21.15.46:40942) 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae4a87640 1 -- 172.21.15.46:0/1365272598 learned_addr learned my addr 172.21.15.46:0/1365272598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae4a87640 1 -- 172.21.15.46:0/1365272598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae019b280 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae4a87640 1 --2- 172.21.15.46:0/1365272598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40090b0 tx=0x7f7ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ad4041070 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad403c7a0 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae019b510 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ad4037040 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.908+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae019e200 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.909+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ad4044070 con 0x7f7ae0106480 2024-09-12T02:39:50.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.909+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ad403d3f0 con 0x7f7ae0106480 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.909+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae0108270 con 0x7f7ae0106480 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.912+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7ad403cc30 con 0x7f7ae0106480 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.973+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7ae002d8a0 con 0x7f7ae0106480 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7addffb640 1 -- 172.21.15.46:0/1365272598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7ad4046450 con 0x7f7ae0106480 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 msgr2=0x7f7ae019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 --2- 172.21.15.46:0/1365272598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40090b0 tx=0x7f7ad40046c0 comp rx=0 tx=0).stop 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 shutdown_connections 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 --2- 172.21.15.46:0/1365272598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ae0106480 0x7f7ae019ad40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 >> 172.21.15.46:0/1365272598 conn(0x7f7ae0101bf0 msgr2=0x7f7ae0191a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 shutdown_connections 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:49.974+0000 7f7ae6ceb640 1 -- 172.21.15.46:0/1365272598 wait complete. 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-09-12T02:39:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T02:39:51.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:51 smithi046 ceph-mon[30583]: from='client.? 172.21.15.46:0/1857217197' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:39:51.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:51 smithi046 ceph-mon[30583]: from='client.? 172.21.15.46:0/1857217197' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T02:39:51.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:51.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:39:51.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:51.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-12T02:39:51.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:39:51.710 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.992+0000 7f017aa2e640 1 Processor -- start 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.992+0000 7f017aa2e640 1 -- start start 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f017aa2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f017aa2e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f017407c8c0 con 0x7f017407de90 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f0173fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f0173fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40958/0 (socket says 172.21.15.46:40958) 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f0173fff640 1 -- 172.21.15.46:0/2780017564 learned_addr learned my addr 172.21.15.46:0/2780017564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.993+0000 7f0173fff640 1 -- 172.21.15.46:0/2780017564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017407ca40 con 0x7f017407de90 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f0173fff640 1 --2- 172.21.15.46:0/2780017564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0164009820 tx=0x7f016402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd30d2eaac45ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f0172ffd640 1 -- 172.21.15.46:0/2780017564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0164037070 con 0x7f017407de90 2024-09-12T02:39:51.711 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f0172ffd640 1 -- 172.21.15.46:0/2780017564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164032440 con 0x7f017407de90 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f0172ffd640 1 -- 172.21.15.46:0/2780017564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0164032740 con 0x7f017407de90 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f017aa2e640 1 -- 172.21.15.46:0/2780017564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 msgr2=0x7f017407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f017aa2e640 1 --2- 172.21.15.46:0/2780017564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0164009820 tx=0x7f016402a270 comp rx=0 tx=0).stop 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f017aa2e640 1 -- 172.21.15.46:0/2780017564 shutdown_connections 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f017aa2e640 1 --2- 172.21.15.46:0/2780017564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017407c2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.994+0000 7f017aa2e640 1 -- 172.21.15.46:0/2780017564 >> 172.21.15.46:0/2780017564 conn(0x7f0174102d40 msgr2=0x7f0174105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 -- 172.21.15.46:0/2780017564 shutdown_connections 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 -- 172.21.15.46:0/2780017564 wait complete. 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 Processor -- start 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 -- start start 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:51.712 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.995+0000 7f017aa2e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f017407cf70 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f0173fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f0173fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40966/0 (socket says 172.21.15.46:40966) 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f0173fff640 1 -- 172.21.15.46:0/1857217197 learned_addr learned my addr 172.21.15.46:0/1857217197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f0173fff640 1 -- 172.21.15.46:0/1857217197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017419d490 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f0173fff640 1 --2- 172.21.15.46:0/1857217197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f01640090b0 tx=0x7f01640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0164041070 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016403c7a0 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017419d720 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0164037040 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a0410 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0164044070 con 0x7f017407de90 2024-09-12T02:39:51.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.996+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f016403d3f0 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.997+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017407c370 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:50.999+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0164046450 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.059+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f017402d910 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.062+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f0164046630 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f01717fa640 1 -- 172.21.15.46:0/1857217197 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f016403d5f0 con 0x7f017407de90 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 msgr2=0x7f017419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 --2- 172.21.15.46:0/1857217197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f01640090b0 tx=0x7f01640046c0 comp rx=0 tx=0).stop 2024-09-12T02:39:51.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 shutdown_connections 2024-09-12T02:39:51.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 --2- 172.21.15.46:0/1857217197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f017407de90 0x7f017419cf50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:51.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 >> 172.21.15.46:0/1857217197 conn(0x7f0174102d40 msgr2=0x7f01741044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:51.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 shutdown_connections 2024-09-12T02:39:51.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:51.063+0000 7f017aa2e640 1 -- 172.21.15.46:0/1857217197 wait complete. 2024-09-12T02:39:51.715 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-09-12T02:39:52.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:52 smithi046 ceph-mon[30583]: from='client.? 172.21.15.46:0/3807806996' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:39:53.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.109+0000 7f808d2ec640 1 Processor -- start 2024-09-12T02:39:53.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.109+0000 7f808d2ec640 1 -- start start 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.109+0000 7f808d2ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.109+0000 7f808d2ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8088107f60 con 0x7f8088107590 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.109+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.110+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40980/0 (socket says 172.21.15.46:40980) 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.110+0000 7f8086ffd640 1 -- 172.21.15.46:0/542837457 learned_addr learned my addr 172.21.15.46:0/542837457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.110+0000 7f8086ffd640 1 -- 172.21.15.46:0/542837457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088108790 con 0x7f8088107590 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f8086ffd640 1 --2- 172.21.15.46:0/542837457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8078009e80 tx=0x7f807802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410a45c60eadc68c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f8085ffb640 1 -- 172.21.15.46:0/542837457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078036070 con 0x7f8088107590 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f8085ffb640 1 -- 172.21.15.46:0/542837457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8078031890 con 0x7f8088107590 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f8085ffb640 1 -- 172.21.15.46:0/542837457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078007980 con 0x7f8088107590 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 -- 172.21.15.46:0/542837457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 msgr2=0x7f8088107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 --2- 172.21.15.46:0/542837457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8078009e80 tx=0x7f807802a7a0 comp rx=0 tx=0).stop 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 -- 172.21.15.46:0/542837457 shutdown_connections 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 --2- 172.21.15.46:0/542837457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f8088107990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 -- 172.21.15.46:0/542837457 >> 172.21.15.46:0/542837457 conn(0x7f8088102d40 msgr2=0x7f8088105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 -- 172.21.15.46:0/542837457 shutdown_connections 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.111+0000 7f808d2ec640 1 -- 172.21.15.46:0/542837457 wait complete. 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.112+0000 7f808d2ec640 1 Processor -- start 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.112+0000 7f808d2ec640 1 -- start start 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.112+0000 7f808d2ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.112+0000 7f808d2ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f808810bb40 con 0x7f8088107590 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40982/0 (socket says 172.21.15.46:40982) 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f8086ffd640 1 -- 172.21.15.46:0/3807806996 learned_addr learned my addr 172.21.15.46:0/3807806996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:53.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f8086ffd640 1 -- 172.21.15.46:0/3807806996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808819ba70 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f8086ffd640 1 --2- 172.21.15.46:0/3807806996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8078007bb0 tx=0x7f8078031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078041020 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f807803c900 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808819bd00 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078036040 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.113+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808819e9f0 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.114+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8078044020 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.114+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80780468e0 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.114+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808810bcc0 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.117+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f807804b020 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.177+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f808819ef70 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f806ffff640 1 -- 172.21.15.46:0/3807806996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f8078046400 con 0x7f8088107590 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 msgr2=0x7f808819b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:53.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f808d2ec640 1 --2- 172.21.15.46:0/3807806996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8078007bb0 tx=0x7f8078031c70 comp rx=0 tx=0).stop 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 shutdown_connections 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f808d2ec640 1 --2- 172.21.15.46:0/3807806996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088107590 0x7f808819b530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.178+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 >> 172.21.15.46:0/3807806996 conn(0x7f8088102d40 msgr2=0x7f8088103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.179+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 shutdown_connections 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:52.179+0000 7f808d2ec640 1 -- 172.21.15.46:0/3807806996 wait complete. 2024-09-12T02:39:53.083 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-09-12T02:39:53.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 systemd[1]: Stopping Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:39:53.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-12T02:39:53.391+0000 7f4979ebb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:39:53.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-12T02:39:53.391+0000 7f4979ebb640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T02:39:53.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 podman[30825]: 2024-09-12 02:39:53.423066747 +0000 UTC m=+0.172478142 container died 59964eb5976823e74b016b5c24907241c4959f879b3c11daddfe58a6b0bfed50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:39:53.959 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 podman[30825]: 2024-09-12 02:39:53.800026817 +0000 UTC m=+0.549438198 container cleanup 59964eb5976823e74b016b5c24907241c4959f879b3c11daddfe58a6b0bfed50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:39:53.960 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 bash[30825]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:39:53.960 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:53 smithi046 podman[30838]: 2024-09-12 02:39:53.958999941 +0000 UTC m=+0.534482480 container remove 59964eb5976823e74b016b5c24907241c4959f879b3c11daddfe58a6b0bfed50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:39:54.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:39:54.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 systemd[1]: Stopped Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:39:54.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 systemd[1]: Starting Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 podman[30932]: 2024-09-12 02:39:54.37672052 +0000 UTC m=+0.106749010 container create 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 podman[30932]: 2024-09-12 02:39:54.286667371 +0000 UTC m=+0.016695831 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 podman[30932]: 2024-09-12 02:39:54.595446701 +0000 UTC m=+0.325475152 container init 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 podman[30932]: 2024-09-12 02:39:54.598635389 +0000 UTC m=+0.328663840 container start 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: pidfile_write: ignore empty --pid-file 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: load: jerasure load: lrc 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Git sha 0 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:39:54.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: DB SUMMARY 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: DB Session ID: HQ8I0EA6K9SG34KHYIBL 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105150 ; 2024-09-12T02:39:54.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:39:54.629 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:39:54.629 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:39:54.629 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.env: 0x557f9e6e3940 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.info_log: 0x557fa0a176e0 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.statistics: (nil) 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.use_fsync: 0 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:39:54.630 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.db_log_dir: 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.wal_dir: 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:39:54.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.write_buffer_manager: 0x557fa0a26dc0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:39:54.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.unordered_write: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.row_cache: None 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.wal_filter: None 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.wal_compression: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:39:54.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:39:54.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_open_files: -1 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Compression algorithms supported: 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kZSTD supported: 0 2024-09-12T02:39:54.635 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.merge_operator: 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_filter: None 2024-09-12T02:39:54.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557fa0a17320) 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:39:54.637 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x557fa0a0b1f0 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-09-12T02:39:54.638 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:39:54.639 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression: NoCompression 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.num_levels: 7 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:39:54.640 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:39:54.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:39:54.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:39:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:39:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.ttl: 2592000 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:39:54.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 502b7fca-cf8d-4230-b0c3-d4c2b29be092 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726108794625551, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726108794626703, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100221, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98110, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14272, "raw_average_key_size": 49, "raw_value_size": 90393, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726108794, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "502b7fca-cf8d-4230-b0c3-d4c2b29be092", "db_session_id": "HQ8I0EA6K9SG34KHYIBL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726108794626802, "job": 1, "event": "recovery_finished"} 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557fa0a38e00 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: DB pointer 0x557fa0b34000 2024-09-12T02:39:54.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-12T02:39:54.647 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 99.73 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 99.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:39:54.648 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 28.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 28.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x557fa0a0b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:39:54.649 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:54.692 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???) e1 preinit fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).mds e1 new map 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).mds e1 print_map 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout: btime 2024-09-12T02:39:49:303263+0000 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:39:54.888 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 ceph-mon[30951]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 bash[30932]: 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 2024-09-12T02:39:54.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:54 smithi046 systemd[1]: Started Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:39:55.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: monmap epoch 1 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: last_changed 2024-09-12T02:39:43.672013+0000 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: min_mon_release 19 (squid) 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: election_strategy: 1 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: fsmap 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: mgrmap e1: no daemons active 2024-09-12T02:39:55.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:55 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1779418045' entity='client.admin' 2024-09-12T02:39:55.713 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.065+0000 7fefb5bda640 1 Processor -- start 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.065+0000 7fefb5bda640 1 -- start start 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.065+0000 7fefb5bda640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.065+0000 7fefb5bda640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0107f60 con 0x7fefb0107590 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.066+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.066+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47542/0 (socket says 172.21.15.46:47542) 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.066+0000 7fefaf7fe640 1 -- 172.21.15.46:0/4281866730 learned_addr learned my addr 172.21.15.46:0/4281866730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.066+0000 7fefaf7fe640 1 -- 172.21.15.46:0/4281866730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0108790 con 0x7fefb0107590 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.068+0000 7fefaf7fe640 1 --2- 172.21.15.46:0/4281866730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009e80 tx=0x7fefa002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5901e5ecb3361f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.068+0000 7fefae7fc640 1 -- 172.21.15.46:0/4281866730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa0036070 con 0x7fefb0107590 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.068+0000 7fefae7fc640 1 -- 172.21.15.46:0/4281866730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fefa0031890 con 0x7fefb0107590 2024-09-12T02:39:55.719 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.068+0000 7fefae7fc640 1 -- 172.21.15.46:0/4281866730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa0007980 con 0x7fefb0107590 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 -- 172.21.15.46:0/4281866730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 msgr2=0x7fefb0107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 --2- 172.21.15.46:0/4281866730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009e80 tx=0x7fefa002a7a0 comp rx=0 tx=0).stop 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 -- 172.21.15.46:0/4281866730 shutdown_connections 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 --2- 172.21.15.46:0/4281866730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb0107990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 -- 172.21.15.46:0/4281866730 >> 172.21.15.46:0/4281866730 conn(0x7fefb0102d40 msgr2=0x7fefb0105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 -- 172.21.15.46:0/4281866730 shutdown_connections 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 -- 172.21.15.46:0/4281866730 wait complete. 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.069+0000 7fefb5bda640 1 Processor -- start 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefb5bda640 1 -- start start 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefb5bda640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefb5bda640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefb010bb40 con 0x7fefb0107590 2024-09-12T02:39:55.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47544/0 (socket says 172.21.15.46:47544) 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefaf7fe640 1 -- 172.21.15.46:0/1779418045 learned_addr learned my addr 172.21.15.46:0/1779418045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefaf7fe640 1 -- 172.21.15.46:0/1779418045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb019ba70 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefaf7fe640 1 --2- 172.21.15.46:0/1779418045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fefa002acc0 tx=0x7fefa0031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa0041030 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fefa003c900 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.070+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb019bd00 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.071+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa0036040 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.071+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb019e9f0 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.071+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fefa0044070 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.071+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefa0042070 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.072+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb010bcc0 con 0x7fefb0107590 2024-09-12T02:39:55.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.074+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fefa004b070 con 0x7fefb0107590 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.134+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fefb019f030 con 0x7fefb0107590 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.135+0000 7fefacff9640 1 -- 172.21.15.46:0/1779418045 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fefa003cab0 con 0x7fefb0107590 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 msgr2=0x7fefb019b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 --2- 172.21.15.46:0/1779418045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fefa002acc0 tx=0x7fefa0031c70 comp rx=0 tx=0).stop 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 shutdown_connections 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 --2- 172.21.15.46:0/1779418045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0107590 0x7fefb019b530 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 >> 172.21.15.46:0/1779418045 conn(0x7fefb0102d40 msgr2=0x7fefb0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 shutdown_connections 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:55.136+0000 7fefb5bda640 1 -- 172.21.15.46:0/1779418045 wait complete. 2024-09-12T02:39:55.722 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T02:39:55.743 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:39:55.743 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-09-12T02:39:55.743 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T02:39:55.744 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T02:39:56.004 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:39:56.008 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T02:39:56.157 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968.target.wants/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@.service. 2024-09-12T02:39:56.282 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 systemd[1]: Starting Ceph mgr.x for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:39:56.690 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 podman[31165]: 2024-09-12 02:39:56.395679227 +0000 UTC m=+0.109749466 container create f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T02:39:56.690 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 podman[31165]: 2024-09-12 02:39:56.302584378 +0000 UTC m=+0.016654587 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:39:56.690 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 podman[31165]: 2024-09-12 02:39:56.61459752 +0000 UTC m=+0.328667715 container init f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T02:39:56.690 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 podman[31165]: 2024-09-12 02:39:56.617297731 +0000 UTC m=+0.331367925 container start f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T02:39:56.690 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 bash[31165]: f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 2024-09-12T02:39:56.706 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:39:56.706 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-12T02:39:56.714 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:39:56.715 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:39:56.715 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-12T02:39:56.715 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:39:56.724 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:39:56.724 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-12T02:39:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:39:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:39:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-12T02:39:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T02:39:56.733 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-09-12T02:39:56.734 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-09-12T02:39:56.967 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 systemd[1]: Started Ceph mgr.x for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:39:56.968 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:56.755+0000 7f1763c09040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:39:56.968 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:56.817+0000 7f1763c09040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:39:57.220 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:56.975+0000 7f1763c09040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:39:57.220 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.045+0000 7f1763c09040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:39:57.221 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.219+0000 7f1763c09040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:39:57.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3830329322' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:39:57.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.382+0000 7f1763c09040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:39:57.896 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.715+0000 7f1763c09040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:39:57.896 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.778+0000 7f1763c09040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:39:57.908 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:39:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:39:49:303263+0000", 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:39:57.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:39:57.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:39:49.303892+0000", 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.186+0000 7fee63c74640 1 Processor -- start 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.186+0000 7fee63c74640 1 -- start start 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee63c74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee63c74640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c108000 con 0x7fee5c107630 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee61a10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee61a10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47568/0 (socket says 172.21.15.46:47568) 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee61a10640 1 -- 172.21.15.46:0/3121274483 learned_addr learned my addr 172.21.15.46:0/3121274483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.187+0000 7fee61a10640 1 -- 172.21.15.46:0/3121274483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c1087e0 con 0x7fee5c107630 2024-09-12T02:39:57.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.188+0000 7fee61a10640 1 --2- 172.21.15.46:0/3121274483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee58009820 tx=0x7fee5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f62e33b0cf58e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.188+0000 7fee60a0e640 1 -- 172.21.15.46:0/3121274483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee58037070 con 0x7fee5c107630 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.188+0000 7fee60a0e640 1 -- 172.21.15.46:0/3121274483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee580328d0 con 0x7fee5c107630 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.188+0000 7fee60a0e640 1 -- 172.21.15.46:0/3121274483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee580307d0 con 0x7fee5c107630 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 -- 172.21.15.46:0/3121274483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 msgr2=0x7fee5c107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 --2- 172.21.15.46:0/3121274483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee58009820 tx=0x7fee5802a270 comp rx=0 tx=0).stop 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 -- 172.21.15.46:0/3121274483 shutdown_connections 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 --2- 172.21.15.46:0/3121274483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c107a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 -- 172.21.15.46:0/3121274483 >> 172.21.15.46:0/3121274483 conn(0x7fee5c102de0 msgr2=0x7fee5c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 -- 172.21.15.46:0/3121274483 shutdown_connections 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.189+0000 7fee63c74640 1 -- 172.21.15.46:0/3121274483 wait complete. 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.190+0000 7fee63c74640 1 Processor -- start 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.190+0000 7fee63c74640 1 -- start start 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee63c74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee63c74640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c10bb90 con 0x7fee5c107630 2024-09-12T02:39:57.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee61a10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee61a10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47574/0 (socket says 172.21.15.46:47574) 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee61a10640 1 -- 172.21.15.46:0/3830329322 learned_addr learned my addr 172.21.15.46:0/3830329322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee61a10640 1 -- 172.21.15.46:0/3830329322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c19ba70 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.191+0000 7fee61a10640 1 --2- 172.21.15.46:0/3830329322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee58009940 tx=0x7fee58032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee58042070 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fee58030c80 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c19bd00 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee58037040 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c19e9f0 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee58045070 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.192+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee58047a40 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.193+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c107ab0 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.195+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fee5803c690 con 0x7fee5c107630 2024-09-12T02:39:57.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.260+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fee5c19ebc0 con 0x7fee5c107630 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.260+0000 7fee52ffd640 1 -- 172.21.15.46:0/3830329322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fee5803c870 con 0x7fee5c107630 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 msgr2=0x7fee5c19b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 --2- 172.21.15.46:0/3830329322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fee58009940 tx=0x7fee58032e10 comp rx=0 tx=0).stop 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 shutdown_connections 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 --2- 172.21.15.46:0/3830329322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee5c107630 0x7fee5c19b530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 >> 172.21.15.46:0/3830329322 conn(0x7fee5c102de0 msgr2=0x7fee5c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 shutdown_connections 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:39:57.261+0000 7fee63c74640 1 -- 172.21.15.46:0/3830329322 wait complete. 2024-09-12T02:39:57.916 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-09-12T02:39:58.282 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:57.977+0000 7f1763c09040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:39:59.282 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:58 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:58.900+0000 7f1763c09040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:39:59.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:58 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:58.970+0000 7f1763c09040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:40:00.000 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:59 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:59.726+0000 7f1763c09040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:40:00.000 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:39:59 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:59.846+0000 7f1763c09040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:40:00.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:39:59.998+0000 7f1763c09040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:40:00.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.063+0000 7f1763c09040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:40:00.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.221+0000 7f1763c09040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:40:00.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2347192002' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:40:00.609 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.294+0000 7f1763c09040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:40:00.609 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.358+0000 7f1763c09040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:40:00.609 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.539+0000 7f1763c09040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:40:00.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.608+0000 7f1763c09040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:40:00.890 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.678+0000 7f1763c09040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:40:00.890 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.765+0000 7f1763c09040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:40:00.890 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.827+0000 7f1763c09040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:40:00.890 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:00.888+0000 7f1763c09040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:40:01.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:40:01.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:01.132 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:40:01.133 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:40:01.134 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:40:01.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:39:49:303263+0000", 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:40:01.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:39:49.303892+0000", 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.319+0000 7fa192cdd640 1 Processor -- start 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa192cdd640 1 -- start start 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa192cdd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:01.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa192cdd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c07b7b0 con 0x7fa18c07cd80 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa190a79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa190a79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47578/0 (socket says 172.21.15.46:47578) 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa190a79640 1 -- 172.21.15.46:0/2797829783 learned_addr learned my addr 172.21.15.46:0/2797829783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.320+0000 7fa190a79640 1 -- 172.21.15.46:0/2797829783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c07b930 con 0x7fa18c07cd80 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.321+0000 7fa190a79640 1 --2- 172.21.15.46:0/2797829783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa180009820 tx=0x7fa18002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efc86bbadf28e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.321+0000 7fa18b7fe640 1 -- 172.21.15.46:0/2797829783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa180037070 con 0x7fa18c07cd80 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.321+0000 7fa18b7fe640 1 -- 172.21.15.46:0/2797829783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa1800328d0 con 0x7fa18c07cd80 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.321+0000 7fa18b7fe640 1 -- 172.21.15.46:0/2797829783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1800307d0 con 0x7fa18c07cd80 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 -- 172.21.15.46:0/2797829783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 msgr2=0x7fa18c07b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 --2- 172.21.15.46:0/2797829783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa180009820 tx=0x7fa18002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 -- 172.21.15.46:0/2797829783 shutdown_connections 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 --2- 172.21.15.46:0/2797829783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c07b1e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 -- 172.21.15.46:0/2797829783 >> 172.21.15.46:0/2797829783 conn(0x7fa18c102d30 msgr2=0x7fa18c105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 -- 172.21.15.46:0/2797829783 shutdown_connections 2024-09-12T02:40:01.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 -- 172.21.15.46:0/2797829783 wait complete. 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.322+0000 7fa192cdd640 1 Processor -- start 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa192cdd640 1 -- start start 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa192cdd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa192cdd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c10db50 con 0x7fa18c07cd80 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa190a79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa190a79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47594/0 (socket says 172.21.15.46:47594) 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa190a79640 1 -- 172.21.15.46:0/2347192002 learned_addr learned my addr 172.21.15.46:0/2347192002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa190a79640 1 -- 172.21.15.46:0/2347192002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c19dc00 con 0x7fa18c07cd80 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa190a79640 1 --2- 172.21.15.46:0/2347192002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1800097f0 tx=0x7fa180030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa180037040 con 0x7fa18c07cd80 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa180032bb0 con 0x7fa18c07cd80 2024-09-12T02:40:01.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c19de90 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.323+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa18003a390 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.324+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c1a0b80 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.324+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa18003a5d0 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.324+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa180044bf0 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.325+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c07b260 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.327+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa180044400 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.389+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa18c1a1100 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.390+0000 7fa189ffb640 1 -- 172.21.15.46:0/2347192002 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fa180032d60 con 0x7fa18c07cd80 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 msgr2=0x7fa18c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 --2- 172.21.15.46:0/2347192002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1800097f0 tx=0x7fa180030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 shutdown_connections 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 --2- 172.21.15.46:0/2347192002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa18c07cd80 0x7fa18c19d6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 >> 172.21.15.46:0/2347192002 conn(0x7fa18c102d30 msgr2=0x7fa18c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 shutdown_connections 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:00.391+0000 7fa192cdd640 1 -- 172.21.15.46:0/2347192002 wait complete. 2024-09-12T02:40:01.146 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:01.718+0000 7f1763c09040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: Activating manager daemon x 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: mgrmap e2: x(active, starting, since 0.00334023s) 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: Manager daemon x is now available 2024-09-12T02:40:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:40:02.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:40:02.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' 2024-09-12T02:40:02.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' 2024-09-12T02:40:02.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:01 smithi046 ceph-mon[30951]: from='mgr.14100 172.21.15.46:0/3663648822' entity='mgr.x' 2024-09-12T02:40:03.817 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:03 smithi046 ceph-mon[30951]: mgrmap e3: x(active, since 1.00704s) 2024-09-12T02:40:04.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:40:04.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:04.372 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:40:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:40:04.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:40:04.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:40:04.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:40:04.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:04.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:39:49:303263+0000", 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:40:04.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:39:49.303892+0000", 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.558+0000 7fbf8ac86640 1 Processor -- start 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.558+0000 7fbf8ac86640 1 -- start start 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf8ac86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf8ac86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84075a80 con 0x7fbf84108020 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf88a22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf88a22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47658/0 (socket says 172.21.15.46:47658) 2024-09-12T02:40:04.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf88a22640 1 -- 172.21.15.46:0/2213528570 learned_addr learned my addr 172.21.15.46:0/2213528570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.559+0000 7fbf88a22640 1 -- 172.21.15.46:0/2213528570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8410a950 con 0x7fbf84108020 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.560+0000 7fbf88a22640 1 --2- 172.21.15.46:0/2213528570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009820 tx=0x7fbf7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9ddd2e8bd37c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.560+0000 7fbf837fe640 1 -- 172.21.15.46:0/2213528570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74037070 con 0x7fbf84108020 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.560+0000 7fbf837fe640 1 -- 172.21.15.46:0/2213528570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbf740328d0 con 0x7fbf84108020 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.560+0000 7fbf837fe640 1 -- 172.21.15.46:0/2213528570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf740307d0 con 0x7fbf84108020 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 -- 172.21.15.46:0/2213528570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 msgr2=0x7fbf8410a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/2213528570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009820 tx=0x7fbf7402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 -- 172.21.15.46:0/2213528570 shutdown_connections 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/2213528570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8410a410 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 -- 172.21.15.46:0/2213528570 >> 172.21.15.46:0/2213528570 conn(0x7fbf84101c60 msgr2=0x7fbf84104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 -- 172.21.15.46:0/2213528570 shutdown_connections 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.561+0000 7fbf8ac86640 1 -- 172.21.15.46:0/2213528570 wait complete. 2024-09-12T02:40:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf8ac86640 1 Processor -- start 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf8ac86640 1 -- start start 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf8ac86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf8ac86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410b990 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf88a22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf88a22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47672/0 (socket says 172.21.15.46:47672) 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf88a22640 1 -- 172.21.15.46:0/4270660646 learned_addr learned my addr 172.21.15.46:0/4270660646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf88a22640 1 -- 172.21.15.46:0/4270660646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8419ba80 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.562+0000 7fbf88a22640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf740097f0 tx=0x7fbf74030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.563+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74037040 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.563+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbf74032bb0 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.563+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8419bd10 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.563+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf7403a390 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.563+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8419ea00 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.564+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fbf7403b3f0 con 0x7fbf84108020 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.564+0000 7fbf81ffb640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 0x7fbf7003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:04.386 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.564+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf84108480 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.564+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf7406bf60 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.564+0000 7fbf83fff640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 0x7fbf7003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.565+0000 7fbf83fff640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 0x7fbf7003b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098c0 tx=0x7fbf6c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.569+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf74032d60 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.729+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbf74037d00 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.816+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbf8419efe0 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.817+0000 7fbf81ffb640 1 -- 172.21.15.46:0/4270660646 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fbf740638b0 con 0x7fbf84108020 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 msgr2=0x7fbf7003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 0x7fbf7003b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098c0 tx=0x7fbf6c006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 msgr2=0x7fbf8419b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf740097f0 tx=0x7fbf74030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 shutdown_connections 2024-09-12T02:40:04.387 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fbf70038e60 0x7fbf7003b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 --2- 172.21.15.46:0/4270660646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf84108020 0x7fbf8419b540 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 >> 172.21.15.46:0/4270660646 conn(0x7fbf84101c60 msgr2=0x7fbf84102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 shutdown_connections 2024-09-12T02:40:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:03.819+0000 7fbf8ac86640 1 -- 172.21.15.46:0/4270660646 wait complete. 2024-09-12T02:40:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-09-12T02:40:04.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:04 smithi046 ceph-mon[30951]: mgrmap e4: x(active, since 2s) 2024-09-12T02:40:04.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:04 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4270660646' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:40:05.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:40:05.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.748+0000 7fdefc47f640 1 Processor -- start 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.748+0000 7fdefc47f640 1 -- start start 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefc47f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefc47f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4075a40 con 0x7fdef4105e50 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefa21b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefa21b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41886/0 (socket says 172.21.15.46:41886) 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefa21b640 1 -- 172.21.15.46:0/3714203058 learned_addr learned my addr 172.21.15.46:0/3714203058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.749+0000 7fdefa21b640 1 -- 172.21.15.46:0/3714203058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef4106790 con 0x7fdef4105e50 2024-09-12T02:40:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefa21b640 1 --2- 172.21.15.46:0/3714203058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009820 tx=0x7fdef002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202511aa42c1bd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdef9219640 1 -- 172.21.15.46:0/3714203058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef0037070 con 0x7fdef4105e50 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdef9219640 1 -- 172.21.15.46:0/3714203058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdef00328d0 con 0x7fdef4105e50 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdef9219640 1 -- 172.21.15.46:0/3714203058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef00307d0 con 0x7fdef4105e50 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 -- 172.21.15.46:0/3714203058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 msgr2=0x7fdef4106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 --2- 172.21.15.46:0/3714203058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009820 tx=0x7fdef002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 -- 172.21.15.46:0/3714203058 shutdown_connections 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 --2- 172.21.15.46:0/3714203058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef4106250 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 -- 172.21.15.46:0/3714203058 >> 172.21.15.46:0/3714203058 conn(0x7fdef4101c60 msgr2=0x7fdef4104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 -- 172.21.15.46:0/3714203058 shutdown_connections 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.750+0000 7fdefc47f640 1 -- 172.21.15.46:0/3714203058 wait complete. 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefc47f640 1 Processor -- start 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefc47f640 1 -- start start 2024-09-12T02:40:05.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefc47f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefc47f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4107830 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefa21b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefa21b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41900/0 (socket says 172.21.15.46:41900) 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.751+0000 7fdefa21b640 1 -- 172.21.15.46:0/532967935 learned_addr learned my addr 172.21.15.46:0/532967935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdefa21b640 1 -- 172.21.15.46:0/532967935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef419dc10 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdefa21b640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdef0032e10 tx=0x7fdef0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef0042070 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef419dea0 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdef0030c80 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef41a0b90 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.752+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef0037040 con 0x7fdef4105e50 2024-09-12T02:40:05.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.753+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fdef0045070 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.753+0000 7fdeeb7fe640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 0x7fdee003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.753+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdef006beb0 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.753+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef4106250 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.753+0000 7fdef9a1a640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 0x7fdee003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.754+0000 7fdef9a1a640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 0x7fdee003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdee40098c0 tx=0x7fdee4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.758+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdef004b170 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.933+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdef41a2300 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.934+0000 7fdeeb7fe640 1 -- 172.21.15.46:0/532967935 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+454 (secure 0 0 0) 0x7fdef006a400 con 0x7fdef4105e50 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 msgr2=0x7fdee003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 0x7fdee003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdee40098c0 tx=0x7fdee4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 msgr2=0x7fdef419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdef0032e10 tx=0x7fdef0032e40 comp rx=0 tx=0).stop 2024-09-12T02:40:05.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 shutdown_connections 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fdee0038f20 0x7fdee003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 --2- 172.21.15.46:0/532967935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef4105e50 0x7fdef419d6d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 >> 172.21.15.46:0/532967935 conn(0x7fdef4101c60 msgr2=0x7fdef4191ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.937+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 shutdown_connections 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:04.938+0000 7fdefc47f640 1 -- 172.21.15.46:0/532967935 wait complete. 2024-09-12T02:40:05.479 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-09-12T02:40:06.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:05 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/532967935' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:40:07.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:06 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2999694485' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T02:40:07.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setuser ceph since I am not root 2024-09-12T02:40:07.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setgroup ceph since I am not root 2024-09-12T02:40:07.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:06.887+0000 7ffbf063d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:40:07.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:06.946+0000 7ffbf063d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:40:07.304 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.104+0000 7ffbf063d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:40:07.304 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.173+0000 7ffbf063d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:40:07.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.859+0000 7f25bda10640 1 Processor -- start 2024-09-12T02:40:07.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.859+0000 7f25bda10640 1 -- start start 2024-09-12T02:40:07.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25bda10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25bda10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25b807c8c0 con 0x7f25b807de90 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41906/0 (socket says 172.21.15.46:41906) 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25b6ffd640 1 -- 172.21.15.46:0/1653966914 learned_addr learned my addr 172.21.15.46:0/1653966914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.860+0000 7f25b6ffd640 1 -- 172.21.15.46:0/1653966914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b807ca40 con 0x7f25b807de90 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.861+0000 7f25b6ffd640 1 --2- 172.21.15.46:0/1653966914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009820 tx=0x7f25a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acb6d2f37aef67f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.861+0000 7f25b5ffb640 1 -- 172.21.15.46:0/1653966914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8037070 con 0x7f25b807de90 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.861+0000 7f25b5ffb640 1 -- 172.21.15.46:0/1653966914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f25a80328d0 con 0x7f25b807de90 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.861+0000 7f25b5ffb640 1 -- 172.21.15.46:0/1653966914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a80307d0 con 0x7f25b807de90 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- 172.21.15.46:0/1653966914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 msgr2=0x7f25b807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 --2- 172.21.15.46:0/1653966914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009820 tx=0x7f25a802a270 comp rx=0 tx=0).stop 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- 172.21.15.46:0/1653966914 shutdown_connections 2024-09-12T02:40:07.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 --2- 172.21.15.46:0/1653966914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b807c2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- 172.21.15.46:0/1653966914 >> 172.21.15.46:0/1653966914 conn(0x7f25b8102d40 msgr2=0x7f25b8105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- 172.21.15.46:0/1653966914 shutdown_connections 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- 172.21.15.46:0/1653966914 wait complete. 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 Processor -- start 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.862+0000 7f25bda10640 1 -- start start 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bda10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bda10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810db60 con 0x7f25b807de90 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41920/0 (socket says 172.21.15.46:41920) 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25b6ffd640 1 -- 172.21.15.46:0/2999694485 learned_addr learned my addr 172.21.15.46:0/2999694485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25b6ffd640 1 -- 172.21.15.46:0/2999694485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b819dc40 con 0x7f25b807de90 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25b6ffd640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25a8032e10 tx=0x7f25a8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8042070 con 0x7f25b807de90 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f25a8030c80 con 0x7f25b807de90 2024-09-12T02:40:07.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b819ded0 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8037040 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.863+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a0bc0 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.864+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f25a8045070 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.864+0000 7f25bca0e640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 0x7f25a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.864+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b807c370 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.864+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25a806bfb0 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.865+0000 7f25b67fc640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 0x7f25a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.865+0000 7f25b67fc640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 0x7f25a003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098c0 tx=0x7f25ac006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:05.869+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25a803b330 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.098+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f25b81a0d90 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.732+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f25b81a0d90 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.734+0000 7f25bca0e640 1 -- 172.21.15.46:0/2999694485 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f25a803c5f0 con 0x7f25b807de90 2024-09-12T02:40:07.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 msgr2=0x7f25a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 0x7f25a003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098c0 tx=0x7f25ac006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 msgr2=0x7f25b819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25a8032e10 tx=0x7f25a8032e40 comp rx=0 tx=0).stop 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 shutdown_connections 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7f25a0038f70 0x7f25a003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 --2- 172.21.15.46:0/2999694485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25b807de90 0x7f25b819d700 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 >> 172.21.15.46:0/2999694485 conn(0x7f25b8102d40 msgr2=0x7f25b81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 shutdown_connections 2024-09-12T02:40:07.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:06.736+0000 7f25bda10640 1 -- 172.21.15.46:0/2999694485 wait complete. 2024-09-12T02:40:07.615 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.330+0000 7ffbf063d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:40:07.615 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.480+0000 7ffbf063d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:40:07.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2999694485' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T02:40:07.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-mon[30951]: mgrmap e5: x(active, since 5s) 2024-09-12T02:40:07.883 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.818+0000 7ffbf063d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:40:07.883 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:07.882+0000 7ffbf063d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:40:08.243 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:08.080+0000 7ffbf063d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:40:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T02:40:08.529 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31eb93640 1 Processor -- start 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31eb93640 1 -- start start 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31eb93640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31eb93640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc318075a20 con 0x7fc318108080 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31c92f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31c92f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41954/0 (socket says 172.21.15.46:41954) 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.694+0000 7fc31c92f640 1 -- 172.21.15.46:0/2844506708 learned_addr learned my addr 172.21.15.46:0/2844506708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.695+0000 7fc31c92f640 1 -- 172.21.15.46:0/2844506708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31810a9b0 con 0x7fc318108080 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.695+0000 7fc31c92f640 1 --2- 172.21.15.46:0/2844506708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009820 tx=0x7fc30c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83582c5896306d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:08.530 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc3177fe640 1 -- 172.21.15.46:0/2844506708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc30c037070 con 0x7fc318108080 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc3177fe640 1 -- 172.21.15.46:0/2844506708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc30c0328d0 con 0x7fc318108080 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc3177fe640 1 -- 172.21.15.46:0/2844506708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc30c0307d0 con 0x7fc318108080 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 -- 172.21.15.46:0/2844506708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 msgr2=0x7fc31810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 --2- 172.21.15.46:0/2844506708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009820 tx=0x7fc30c02a270 comp rx=0 tx=0).stop 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 -- 172.21.15.46:0/2844506708 shutdown_connections 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 --2- 172.21.15.46:0/2844506708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc31810a470 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 -- 172.21.15.46:0/2844506708 >> 172.21.15.46:0/2844506708 conn(0x7fc318101c60 msgr2=0x7fc318104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 -- 172.21.15.46:0/2844506708 shutdown_connections 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.696+0000 7fc31eb93640 1 -- 172.21.15.46:0/2844506708 wait complete. 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31eb93640 1 Processor -- start 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31eb93640 1 -- start start 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31eb93640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31eb93640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc31810b990 con 0x7fc318108080 2024-09-12T02:40:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31c92f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31c92f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41970/0 (socket says 172.21.15.46:41970) 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.697+0000 7fc31c92f640 1 -- 172.21.15.46:0/585435733 learned_addr learned my addr 172.21.15.46:0/585435733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc31c92f640 1 -- 172.21.15.46:0/585435733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318107e40 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc31c92f640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0097f0 tx=0x7fc30c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc30c037040 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc30c032bb0 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc318106030 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc30c03a390 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.698+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc318106570 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc30c03b3f0 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3181084e0 con 0x7fc318108080 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc315ffb640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 0x7fc30803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:08.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc30c06c110 con 0x7fc318108080 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc317fff640 1 -- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 msgr2=0x7fc30803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.699+0000 7fc317fff640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 0x7fc30803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.704+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc30c040210 con 0x7fc318108080 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.900+0000 7fc317fff640 1 -- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 msgr2=0x7fc30803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.900+0000 7fc317fff640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 0x7fc30803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.931+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc3181067f0 con 0x7fc318108080 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.932+0000 7fc315ffb640 1 -- 172.21.15.46:0/585435733 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc30c038d40 con 0x7fc318108080 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 msgr2=0x7fc30803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 0x7fc30803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 msgr2=0x7fc318107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0097f0 tx=0x7fc30c030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 shutdown_connections 2024-09-12T02:40:08.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fc308038fc0 0x7fc30803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 --2- 172.21.15.46:0/585435733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc318108080 0x7fc318107900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 >> 172.21.15.46:0/585435733 conn(0x7fc318101c60 msgr2=0x7fc318102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 shutdown_connections 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:07.934+0000 7fc31eb93640 1 -- 172.21.15.46:0/585435733 wait complete. 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-12T02:40:08.534 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-09-12T02:40:08.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:08 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/585435733' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:40:09.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:09.033+0000 7ffbf063d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:40:09.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:09.105+0000 7ffbf063d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:40:10.151 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:09.875+0000 7ffbf063d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:40:10.152 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:09.998+0000 7ffbf063d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:40:10.444 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.150+0000 7ffbf063d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:40:10.444 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.215+0000 7ffbf063d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:40:10.445 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.372+0000 7ffbf063d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:40:10.445 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.444+0000 7ffbf063d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:40:10.741 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.507+0000 7ffbf063d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:40:10.741 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.672+0000 7ffbf063d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:40:10.741 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.741+0000 7ffbf063d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:40:11.022 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.811+0000 7ffbf063d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:40:11.023 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.898+0000 7ffbf063d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:40:11.023 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:10.960+0000 7ffbf063d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:40:11.023 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:11.022+0000 7ffbf063d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:40:12.282 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:11.853+0000 7ffbf063d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: Active manager daemon x restarted 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: Activating manager daemon x 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: mgrmap e6: x(active, starting, since 0.00346627s) 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:40:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: Manager daemon x is now available 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:40:12.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:11 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:40:13.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:12 smithi046 ceph-mon[30951]: Found migration_current of "None". Setting to last migration. 2024-09-12T02:40:13.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:12 smithi046 ceph-mon[30951]: mgrmap e7: x(active, since 1.00547s) 2024-09-12T02:40:13.434 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.909+0000 7fa983764640 1 Processor -- start 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa983764640 1 -- start start 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa983764640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa983764640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c075a80 con 0x7fa97c108010 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa981500640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa981500640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41974/0 (socket says 172.21.15.46:41974) 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa981500640 1 -- 172.21.15.46:0/3765664856 learned_addr learned my addr 172.21.15.46:0/3765664856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.910+0000 7fa981500640 1 -- 172.21.15.46:0/3765664856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c10a940 con 0x7fa97c108010 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.911+0000 7fa981500640 1 --2- 172.21.15.46:0/3765664856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa978009820 tx=0x7fa97802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4504b557dc297f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.911+0000 7fa973fff640 1 -- 172.21.15.46:0/3765664856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa978037070 con 0x7fa97c108010 2024-09-12T02:40:13.435 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.911+0000 7fa973fff640 1 -- 172.21.15.46:0/3765664856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa9780328d0 con 0x7fa97c108010 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.911+0000 7fa973fff640 1 -- 172.21.15.46:0/3765664856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9780307d0 con 0x7fa97c108010 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.912+0000 7fa983764640 1 -- 172.21.15.46:0/3765664856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 msgr2=0x7fa97c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.912+0000 7fa983764640 1 --2- 172.21.15.46:0/3765664856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa978009820 tx=0x7fa97802a270 comp rx=0 tx=0).stop 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.912+0000 7fa983764640 1 -- 172.21.15.46:0/3765664856 shutdown_connections 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.912+0000 7fa983764640 1 --2- 172.21.15.46:0/3765664856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c10a400 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.912+0000 7fa983764640 1 -- 172.21.15.46:0/3765664856 >> 172.21.15.46:0/3765664856 conn(0x7fa97c101c30 msgr2=0x7fa97c104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.913+0000 7fa983764640 1 -- 172.21.15.46:0/3765664856 shutdown_connections 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.913+0000 7fa983764640 1 -- 172.21.15.46:0/3765664856 wait complete. 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.913+0000 7fa983764640 1 Processor -- start 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.913+0000 7fa983764640 1 -- start start 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa983764640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa983764640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10b980 con 0x7fa97c108010 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa981500640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa981500640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41986/0 (socket says 172.21.15.46:41986) 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa981500640 1 -- 172.21.15.46:0/3140294485 learned_addr learned my addr 172.21.15.46:0/3140294485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa981500640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c19ba40 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.914+0000 7fa981500640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097f0 tx=0x7fa978030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.915+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa978037040 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.915+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa978032bb0 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.915+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa97803a390 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.915+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c19bcd0 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.915+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c19e9c0 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa97803a5d0 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa9727fc640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa96803bb40 con 0x7fa968038f70 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa97806c160 con 0x7fa97c108010 2024-09-12T02:40:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa980cff640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 msgr2=0x7fa96803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:08.916+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:09.117+0000 7fa980cff640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 msgr2=0x7fa96803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:09.117+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:09.518+0000 7fa980cff640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 msgr2=0x7fa96803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:09.518+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:10.319+0000 7fa980cff640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 msgr2=0x7fa96803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4229481015 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:10.319+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:11.857+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fa97806ace0 con 0x7fa97c108010 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:11.857+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 msgr2=0x7fa96803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:11.857+0000 7fa9727fc640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.858+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fa97803a8c0 con 0x7fa97c108010 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.858+0000 7fa9727fc640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 0x7fa96803ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.858+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa978001d30 con 0x7fa96803c8a0 2024-09-12T02:40:13.438 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.859+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 0x7fa96803ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.859+0000 7fa980cff640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 0x7fa96803ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa96c003980 tx=0x7fa96c006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.860+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa978001d30 con 0x7fa96803c8a0 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa97c108470 con 0x7fa96803c8a0 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa9727fc640 1 -- 172.21.15.46:0/3140294485 <== mgr.14118 v2:172.21.15.46:6800/4279295285 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa97c108470 con 0x7fa96803c8a0 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 msgr2=0x7fa96803ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 0x7fa96803ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa96c003980 tx=0x7fa96c006e10 comp rx=0 tx=0).stop 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 msgr2=0x7fa97c19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097f0 tx=0x7fa978030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 shutdown_connections 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fa96803c8a0 0x7fa96803ec90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:6800/4229481015,v1:172.21.15.46:6801/4229481015] conn(0x7fa968038f70 0x7fa96803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 --2- 172.21.15.46:0/3140294485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97c108010 0x7fa97c19b500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.867+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 >> 172.21.15.46:0/3140294485 conn(0x7fa97c101c30 msgr2=0x7fa97c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:13.439 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.868+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 shutdown_connections 2024-09-12T02:40:13.440 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:12.868+0000 7fa983764640 1 -- 172.21.15.46:0/3140294485 wait complete. 2024-09-12T02:40:13.440 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-09-12T02:40:13.440 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.884+0000 7fb59591b640 1 Processor -- start 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.884+0000 7fb59591b640 1 -- start start 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.884+0000 7fb59591b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.884+0000 7fb59591b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb59007be40 con 0x7fb590108070 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.885+0000 7fb58effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.885+0000 7fb58effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42028/0 (socket says 172.21.15.46:42028) 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.885+0000 7fb58effd640 1 -- 172.21.15.46:0/2210397338 learned_addr learned my addr 172.21.15.46:0/2210397338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.885+0000 7fb58effd640 1 -- 172.21.15.46:0/2210397338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59010a9a0 con 0x7fb590108070 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb58effd640 1 --2- 172.21.15.46:0/2210397338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb580009820 tx=0x7fb58002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ef3bdec4cd8242 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb58e7fc640 1 -- 172.21.15.46:0/2210397338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb590108070 2024-09-12T02:40:14.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb58e7fc640 1 -- 172.21.15.46:0/2210397338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb5800328d0 con 0x7fb590108070 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb58e7fc640 1 -- 172.21.15.46:0/2210397338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5800307d0 con 0x7fb590108070 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb59591b640 1 -- 172.21.15.46:0/2210397338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 msgr2=0x7fb59010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb59591b640 1 --2- 172.21.15.46:0/2210397338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb580009820 tx=0x7fb58002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb59591b640 1 -- 172.21.15.46:0/2210397338 shutdown_connections 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.886+0000 7fb59591b640 1 --2- 172.21.15.46:0/2210397338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59010a460 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 -- 172.21.15.46:0/2210397338 >> 172.21.15.46:0/2210397338 conn(0x7fb590101bf0 msgr2=0x7fb590104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 -- 172.21.15.46:0/2210397338 shutdown_connections 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 -- 172.21.15.46:0/2210397338 wait complete. 2024-09-12T02:40:14.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 Processor -- start 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 -- start start 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb59591b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010b920 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.887+0000 7fb58effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb58effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42036/0 (socket says 172.21.15.46:42036) 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb58effd640 1 -- 172.21.15.46:0/2630456662 learned_addr learned my addr 172.21.15.46:0/2630456662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb58effd640 1 -- 172.21.15.46:0/2630456662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019dc10 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb58effd640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097f0 tx=0x7fb580030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb580037040 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb580032bb0 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019dea0 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a0b90 con 0x7fb590108070 2024-09-12T02:40:14.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.888+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb58003a390 con 0x7fb590108070 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.889+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fb58003a600 con 0x7fb590108070 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.889+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5901084d0 con 0x7fb590108070 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.889+0000 7fb594919640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 0x7fb57c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.889+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb58006c010 con 0x7fb590108070 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.890+0000 7fb587fff640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 0x7fb57c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.890+0000 7fb587fff640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 0x7fb57c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098c0 tx=0x7fb578006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:13.894+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb580032d60 con 0x7fb590108070 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.083+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb590102e00 con 0x7fb57c038eb0 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.088+0000 7fb594919640 1 -- 172.21.15.46:0/2630456662 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb590102e00 con 0x7fb57c038eb0 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 msgr2=0x7fb57c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 0x7fb57c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098c0 tx=0x7fb578006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 msgr2=0x7fb59019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097f0 tx=0x7fb580030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 shutdown_connections 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb57c038eb0 0x7fb57c03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 --2- 172.21.15.46:0/2630456662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb590108070 0x7fb59019d6d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.090+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 >> 172.21.15.46:0/2630456662 conn(0x7fb590101bf0 msgr2=0x7fb5901026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.091+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 shutdown_connections 2024-09-12T02:40:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:14.091+0000 7fb59591b640 1 -- 172.21.15.46:0/2630456662 wait complete. 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:14.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:14 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:15.461+0000 7ffb5d162640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:40:15] ENGINE Error in HTTPServer.serve 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Traceback (most recent call last): 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._connections.run(self.expiration_interval) 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._run(expiration_interval) 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:40:15.721 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s = self.context.wrap_socket( 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: return self.sslsocket_class._create( 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self.do_handshake() 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._sslobj.do_handshake() 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:40:15.722 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:15.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T02:40:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.003+0000 7fbfcecbb640 1 Processor -- start 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.003+0000 7fbfcecbb640 1 -- start start 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcecbb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcecbb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8075a20 con 0x7fbfc8108070 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcca57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcca57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49300/0 (socket says 172.21.15.46:49300) 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcca57640 1 -- 172.21.15.46:0/817829371 learned_addr learned my addr 172.21.15.46:0/817829371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.004+0000 7fbfcca57640 1 -- 172.21.15.46:0/817829371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc810a9a0 con 0x7fbfc8108070 2024-09-12T02:40:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.005+0000 7fbfcca57640 1 --2- 172.21.15.46:0/817829371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009820 tx=0x7fbfc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b15e248871edfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.005+0000 7fbfbf7fe640 1 -- 172.21.15.46:0/817829371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc4037070 con 0x7fbfc8108070 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.005+0000 7fbfbf7fe640 1 -- 172.21.15.46:0/817829371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbfc40328d0 con 0x7fbfc8108070 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.005+0000 7fbfbf7fe640 1 -- 172.21.15.46:0/817829371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc40307d0 con 0x7fbfc8108070 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- 172.21.15.46:0/817829371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 msgr2=0x7fbfc810a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/817829371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009820 tx=0x7fbfc402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- 172.21.15.46:0/817829371 shutdown_connections 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/817829371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc810a460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- 172.21.15.46:0/817829371 >> 172.21.15.46:0/817829371 conn(0x7fbfc8101c30 msgr2=0x7fbfc8104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- 172.21.15.46:0/817829371 shutdown_connections 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- 172.21.15.46:0/817829371 wait complete. 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 Processor -- start 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- start start 2024-09-12T02:40:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.006+0000 7fbfcecbb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810b980 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcca57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcca57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49306/0 (socket says 172.21.15.46:49306) 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcca57640 1 -- 172.21.15.46:0/390756877 learned_addr learned my addr 172.21.15.46:0/390756877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcca57640 1 -- 172.21.15.46:0/390756877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc819ba40 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcca57640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40097f0 tx=0x7fbfc4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc4037040 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbfc4032bb0 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc819bcd0 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc403a390 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.007+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc819e9c0 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.008+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbfc403a580 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.008+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc81084d0 con 0x7fbfc8108070 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.008+0000 7fbfbdffb640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 0x7fbfb003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.008+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbfc406c1e0 con 0x7fbfc8108070 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.009+0000 7fbfbffff640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 0x7fbfb003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.009+0000 7fbfbffff640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 0x7fbfb003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098c0 tx=0x7fbfb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.013+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfc403ade0 con 0x7fbfc8108070 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.193+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fbfc8102ea0 con 0x7fbfb0038fc0 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.193+0000 7fbfbdffb640 1 -- 172.21.15.46:0/390756877 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fbfc8102ea0 con 0x7fbfb0038fc0 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.195+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 msgr2=0x7fbfb003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.195+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 0x7fbfb003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098c0 tx=0x7fbfb8006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 msgr2=0x7fbfc819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40097f0 tx=0x7fbfc4030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 shutdown_connections 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fbfb0038fc0 0x7fbfb003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 --2- 172.21.15.46:0/390756877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8108070 0x7fbfc819b500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 >> 172.21.15.46:0/390756877 conn(0x7fbfc8101c30 msgr2=0x7fbfc8102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 shutdown_connections 2024-09-12T02:40:15.731 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:15.196+0000 7fbfcecbb640 1 -- 172.21.15.46:0/390756877 wait complete. 2024-09-12T02:40:15.732 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-09-12T02:40:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-mon[30951]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-mon[30951]: mgrmap e8: x(active, since 2s) 2024-09-12T02:40:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:15 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:16.726 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Generating public/private rsa key pair. 2024-09-12T02:40:16.726 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Your identification has been saved in /tmp/tmpf1_ztooo/key 2024-09-12T02:40:16.726 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Your public key has been saved in /tmp/tmpf1_ztooo/key.pub 2024-09-12T02:40:16.726 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: The key fingerprint is: 2024-09-12T02:40:16.726 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: SHA256:6hz4XA0HjxQbEmr9w3JSOKMSRJqQ9MCV/5U8K11LWRM ceph-2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: The key's randomart image is: 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: +---[RSA 3072]----+ 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: |+=+.. o.o E. | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: |o=o. o o + o | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: |o ..+ = * . o . | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | o o B O + | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | . . + S B . | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | . . B B . | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | . o o . | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | = o | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: | = | 2024-09-12T02:40:16.727 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: +----[SHA256]-----+ 2024-09-12T02:40:16.985 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:16.985 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:15] ENGINE Bus STARTING 2024-09-12T02:40:16.985 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:15] ENGINE Serving on https://172.21.15.46:7150 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:15] ENGINE Error in HTTPServer.serve 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:40:16.986 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:15] ENGINE Serving on http://172.21.15.46:8765 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:15] ENGINE Bus STARTED 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:16.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:16 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:16.992 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57c0b53640 1 Processor -- start 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57c0b53640 1 -- start start 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57c0b53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57c0b53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc107f60 con 0x7f57bc107590 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49308/0 (socket says 172.21.15.46:49308) 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.138+0000 7f57ba59c640 1 -- 172.21.15.46:0/195217676 learned_addr learned my addr 172.21.15.46:0/195217676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.139+0000 7f57ba59c640 1 -- 172.21.15.46:0/195217676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc108790 con 0x7f57bc107590 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.139+0000 7f57ba59c640 1 --2- 172.21.15.46:0/195217676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009820 tx=0x7f57ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca2d582fa551613 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.139+0000 7f57b959a640 1 -- 172.21.15.46:0/195217676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac037070 con 0x7f57bc107590 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.139+0000 7f57b959a640 1 -- 172.21.15.46:0/195217676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f57ac0328d0 con 0x7f57bc107590 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.139+0000 7f57b959a640 1 -- 172.21.15.46:0/195217676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac0307d0 con 0x7f57bc107590 2024-09-12T02:40:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 -- 172.21.15.46:0/195217676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 msgr2=0x7f57bc107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 --2- 172.21.15.46:0/195217676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009820 tx=0x7f57ac02a270 comp rx=0 tx=0).stop 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 -- 172.21.15.46:0/195217676 shutdown_connections 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 --2- 172.21.15.46:0/195217676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc107990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 -- 172.21.15.46:0/195217676 >> 172.21.15.46:0/195217676 conn(0x7f57bc102d40 msgr2=0x7f57bc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 -- 172.21.15.46:0/195217676 shutdown_connections 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.140+0000 7f57c0b53640 1 -- 172.21.15.46:0/195217676 wait complete. 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57c0b53640 1 Processor -- start 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57c0b53640 1 -- start start 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57c0b53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57c0b53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10bb40 con 0x7f57bc107590 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49314/0 (socket says 172.21.15.46:49314) 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57ba59c640 1 -- 172.21.15.46:0/3089184595 learned_addr learned my addr 172.21.15.46:0/3089184595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:16.994 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.141+0000 7f57ba59c640 1 -- 172.21.15.46:0/3089184595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc07e0c0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57ba59c640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097f0 tx=0x7f57ac030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac037040 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc07c2b0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f57ac032bb0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac03a390 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.142+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc07c7f0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.143+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f57ac03a5d0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.143+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57bc107a10 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.143+0000 7f57ab7fe640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 0x7f57a403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.143+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f57ac06c310 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.143+0000 7f57b9d9b640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 0x7f57a403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.144+0000 7f57b9d9b640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 0x7f57a403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098c0 tx=0x7f57b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.148+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57ac0313a0 con 0x7f57bc107590 2024-09-12T02:40:16.995 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.325+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f57bc103f60 con 0x7f57a4038f70 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.431+0000 7f57ab7fe640 1 -- 172.21.15.46:0/3089184595 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57bc103f60 con 0x7f57a4038f70 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 msgr2=0x7f57a403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 0x7f57a403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098c0 tx=0x7f57b0006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 msgr2=0x7f57bc07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097f0 tx=0x7f57ac030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 shutdown_connections 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f57a4038f70 0x7f57a403b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 --2- 172.21.15.46:0/3089184595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57bc107590 0x7f57bc07db80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.433+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 >> 172.21.15.46:0/3089184595 conn(0x7f57bc102d40 msgr2=0x7f57bc103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.434+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 shutdown_connections 2024-09-12T02:40:16.996 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:16.434+0000 7f57c0b53640 1 -- 172.21.15.46:0/3089184595 wait complete. 2024-09-12T02:40:17.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:17 smithi046 ceph-mon[30951]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:17.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:17 smithi046 ceph-mon[30951]: Generating ssh key... 2024-09-12T02:40:18.172 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35dd7fa640 1 Processor -- start 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35dd7fa640 1 -- start start 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35dd7fa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35dd7fa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8107f60 con 0x7f35d8107590 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49330/0 (socket says 172.21.15.46:49330) 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35d6ffd640 1 -- 172.21.15.46:0/1402482872 learned_addr learned my addr 172.21.15.46:0/1402482872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.369+0000 7f35d6ffd640 1 -- 172.21.15.46:0/1402482872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d8108790 con 0x7f35d8107590 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.370+0000 7f35d6ffd640 1 --2- 172.21.15.46:0/1402482872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009820 tx=0x7f35c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d75a53c1790cd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.370+0000 7f35d5ffb640 1 -- 172.21.15.46:0/1402482872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c8037070 con 0x7f35d8107590 2024-09-12T02:40:18.173 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.370+0000 7f35d5ffb640 1 -- 172.21.15.46:0/1402482872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f35c80328d0 con 0x7f35d8107590 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.370+0000 7f35d5ffb640 1 -- 172.21.15.46:0/1402482872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c80307d0 con 0x7f35d8107590 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 -- 172.21.15.46:0/1402482872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 msgr2=0x7f35d8107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/1402482872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009820 tx=0x7f35c802a270 comp rx=0 tx=0).stop 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 -- 172.21.15.46:0/1402482872 shutdown_connections 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/1402482872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d8107990 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 -- 172.21.15.46:0/1402482872 >> 172.21.15.46:0/1402482872 conn(0x7f35d8102d40 msgr2=0x7f35d8105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 -- 172.21.15.46:0/1402482872 shutdown_connections 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 -- 172.21.15.46:0/1402482872 wait complete. 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.371+0000 7f35dd7fa640 1 Processor -- start 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35dd7fa640 1 -- start start 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35dd7fa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:18.174 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35dd7fa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35d810bb40 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49346/0 (socket says 172.21.15.46:49346) 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35d6ffd640 1 -- 172.21.15.46:0/78373953 learned_addr learned my addr 172.21.15.46:0/78373953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35d6ffd640 1 -- 172.21.15.46:0/78373953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d807e0c0 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.372+0000 7f35d6ffd640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097f0 tx=0x7f35c8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c8037040 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f35c8032bb0 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d807c2b0 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c803a390 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d807c7f0 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.373+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f35c803b3f0 con 0x7f35d8107590 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.374+0000 7f35cffff640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 0x7f35c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:18.175 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.374+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f35c806c150 con 0x7f35d8107590 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.374+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8107a10 con 0x7f35d8107590 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.374+0000 7f35d67fc640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 0x7f35c403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.374+0000 7f35d67fc640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 0x7f35c403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098c0 tx=0x7f35c0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.379+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35c8040210 con 0x7f35d8107590 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.554+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f35d8103f60 con 0x7f35c4038fc0 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.555+0000 7f35cffff640 1 -- 172.21.15.46:0/78373953 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f35d8103f60 con 0x7f35c4038fc0 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.557+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 msgr2=0x7f35c403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.557+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 0x7f35c403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098c0 tx=0x7f35c0006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.557+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 msgr2=0x7f35d807db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.557+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097f0 tx=0x7f35c8030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 shutdown_connections 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f35c4038fc0 0x7f35c403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 --2- 172.21.15.46:0/78373953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d8107590 0x7f35d807db80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:18.177 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 >> 172.21.15.46:0/78373953 conn(0x7f35d8102d40 msgr2=0x7f35d8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:18.177 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 shutdown_connections 2024-09-12T02:40:18.177 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:17.558+0000 7f35dd7fa640 1 -- 172.21.15.46:0/78373953 wait complete. 2024-09-12T02:40:18.177 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:40:18.177 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T02:40:18.179 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-09-12T02:40:18.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:18 smithi046 ceph-mon[30951]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:19.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:19 smithi046 ceph-mon[30951]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:21.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:20 smithi046 ceph-mon[30951]: Deploying cephadm binary to smithi046 2024-09-12T02:40:23.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe0819e640 1 Processor -- start 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe0819e640 1 -- start start 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe0819e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe0819e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe0007be40 con 0x7efe00105e60 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe05f3a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe05f3a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49348/0 (socket says 172.21.15.46:49348) 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.565+0000 7efe05f3a640 1 -- 172.21.15.46:0/187894974 learned_addr learned my addr 172.21.15.46:0/187894974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.566+0000 7efe05f3a640 1 -- 172.21.15.46:0/187894974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001067a0 con 0x7efe00105e60 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.566+0000 7efe05f3a640 1 --2- 172.21.15.46:0/187894974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009820 tx=0x7efdfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480e266f11c33998 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe04f38640 1 -- 172.21.15.46:0/187894974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdfc037070 con 0x7efe00105e60 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe04f38640 1 -- 172.21.15.46:0/187894974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efdfc0328d0 con 0x7efe00105e60 2024-09-12T02:40:23.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe04f38640 1 -- 172.21.15.46:0/187894974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdfc0307d0 con 0x7efe00105e60 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe0819e640 1 -- 172.21.15.46:0/187894974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 msgr2=0x7efe00106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe0819e640 1 --2- 172.21.15.46:0/187894974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009820 tx=0x7efdfc02a270 comp rx=0 tx=0).stop 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe0819e640 1 -- 172.21.15.46:0/187894974 shutdown_connections 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe0819e640 1 --2- 172.21.15.46:0/187894974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe00106260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.567+0000 7efe0819e640 1 -- 172.21.15.46:0/187894974 >> 172.21.15.46:0/187894974 conn(0x7efe00101bf0 msgr2=0x7efe00104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 -- 172.21.15.46:0/187894974 shutdown_connections 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 -- 172.21.15.46:0/187894974 wait complete. 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 Processor -- start 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 -- start start 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.568+0000 7efe0819e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe00107840 con 0x7efe00105e60 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe05f3a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:23.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe05f3a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49364/0 (socket says 172.21.15.46:49364) 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe05f3a640 1 -- 172.21.15.46:0/3987633926 learned_addr learned my addr 172.21.15.46:0/3987633926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe05f3a640 1 -- 172.21.15.46:0/3987633926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0019dbc0 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe05f3a640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097f0 tx=0x7efdfc030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdfc037040 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efdfc032bb0 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0019de50 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdfc03a390 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.569+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a0b40 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.570+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efdfc03a5d0 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.570+0000 7efdf6ffd640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 0x7efdec03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.570+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efdfc06c210 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.570+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00106260 con 0x7efe00105e60 2024-09-12T02:40:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.571+0000 7efe05739640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 0x7efdec03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.571+0000 7efe05739640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 0x7efdec03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098c0 tx=0x7efdf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.575+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdfc040210 con 0x7efe00105e60 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:18.755+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) -- 0x7efe00062630 con 0x7efdec038f70 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.714+0000 7efdf6ffd640 1 -- 172.21.15.46:0/3987633926 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7efe00062630 con 0x7efdec038f70 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 msgr2=0x7efdec03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 0x7efdec03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098c0 tx=0x7efdf0006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 msgr2=0x7efe0019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097f0 tx=0x7efdfc030d00 comp rx=0 tx=0).stop 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 shutdown_connections 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7efdec038f70 0x7efdec03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 --2- 172.21.15.46:0/3987633926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00105e60 0x7efe0019d680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 >> 172.21.15.46:0/3987633926 conn(0x7efe00101bf0 msgr2=0x7efe00191a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 shutdown_connections 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:22.718+0000 7efe0819e640 1 -- 172.21.15.46:0/3987633926 wait complete. 2024-09-12T02:40:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-09-12T02:40:24.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:23 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:24.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:23 smithi046 ceph-mon[30951]: Added host smithi046 2024-09-12T02:40:24.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:23 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:25 smithi046 ceph-mon[30951]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:25 smithi046 ceph-mon[30951]: Saving service mon spec with placement count:5 2024-09-12T02:40:25.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:25 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:25.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T02:40:25.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.793+0000 7f076efae640 1 Processor -- start 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.793+0000 7f076efae640 1 -- start start 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076efae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076efae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0768075a40 con 0x7f0768108060 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076cd4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076cd4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49376/0 (socket says 172.21.15.46:49376) 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076cd4a640 1 -- 172.21.15.46:0/3693221703 learned_addr learned my addr 172.21.15.46:0/3693221703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.794+0000 7f076cd4a640 1 -- 172.21.15.46:0/3693221703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076810a990 con 0x7f0768108060 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.795+0000 7f076cd4a640 1 --2- 172.21.15.46:0/3693221703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0758009e80 tx=0x7f075802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344854c0a047ba12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.795+0000 7f07677fe640 1 -- 172.21.15.46:0/3693221703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0758036070 con 0x7f0768108060 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.795+0000 7f07677fe640 1 -- 172.21.15.46:0/3693221703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0758031890 con 0x7f0768108060 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.795+0000 7f07677fe640 1 -- 172.21.15.46:0/3693221703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0758007980 con 0x7f0768108060 2024-09-12T02:40:25.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- 172.21.15.46:0/3693221703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 msgr2=0x7f076810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 --2- 172.21.15.46:0/3693221703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0758009e80 tx=0x7f075802a7a0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- 172.21.15.46:0/3693221703 shutdown_connections 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 --2- 172.21.15.46:0/3693221703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076810a450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- 172.21.15.46:0/3693221703 >> 172.21.15.46:0/3693221703 conn(0x7f0768101c60 msgr2=0x7f0768104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- 172.21.15.46:0/3693221703 shutdown_connections 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- 172.21.15.46:0/3693221703 wait complete. 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 Processor -- start 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 -- start start 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.796+0000 7f076efae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076efae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f076810b970 con 0x7f0768108060 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076cd4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076cd4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49386/0 (socket says 172.21.15.46:49386) 2024-09-12T02:40:25.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076cd4a640 1 -- 172.21.15.46:0/1740713792 learned_addr learned my addr 172.21.15.46:0/1740713792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076cd4a640 1 -- 172.21.15.46:0/1740713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076819ba20 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076cd4a640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0758031c70 tx=0x7f0758031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0758041020 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f075803c900 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0758036040 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.797+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076819bcb0 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.798+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076819e9a0 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.798+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0758044020 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.798+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07681084c0 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.798+0000 7f0765ffb640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 0x7f075403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.799+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f075806d450 con 0x7f0768108060 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.799+0000 7f0767fff640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 0x7f075403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:25.599 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.799+0000 7f0767fff640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 0x7f075403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f07500098c0 tx=0x7f0750006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:23.803+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f075804b020 con 0x7f0768108060 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.002+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0768102ec0 con 0x7f0754038f70 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.007+0000 7f0765ffb640 1 -- 172.21.15.46:0/1740713792 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0768102ec0 con 0x7f0754038f70 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 msgr2=0x7f075403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 0x7f075403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f07500098c0 tx=0x7f0750006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 msgr2=0x7f076819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0758031c70 tx=0x7f0758031ca0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 shutdown_connections 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f0754038f70 0x7f075403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 --2- 172.21.15.46:0/1740713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768108060 0x7f076819b4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.009+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 >> 172.21.15.46:0/1740713792 conn(0x7f0768101c60 msgr2=0x7f07681027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.010+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 shutdown_connections 2024-09-12T02:40:25.600 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:24.010+0000 7f076efae640 1 -- 172.21.15.46:0/1740713792 wait complete. 2024-09-12T02:40:25.601 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-09-12T02:40:26.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:26 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:26.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:26 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:27.233 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3b0493640 1 Processor -- start 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3b0493640 1 -- start start 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3b0493640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3b0493640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81084e0 con 0x7ff3a8107b10 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3ae22f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3ae22f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56808/0 (socket says 172.21.15.46:56808) 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.991+0000 7ff3ae22f640 1 -- 172.21.15.46:0/3693987265 learned_addr learned my addr 172.21.15.46:0/3693987265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.992+0000 7ff3ae22f640 1 -- 172.21.15.46:0/3693987265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8108660 con 0x7ff3a8107b10 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.992+0000 7ff3ae22f640 1 --2- 172.21.15.46:0/3693987265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009820 tx=0x7ff3a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc2b70669f1fbfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.992+0000 7ff3ad22d640 1 -- 172.21.15.46:0/3693987265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3a4037070 con 0x7ff3a8107b10 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.992+0000 7ff3ad22d640 1 -- 172.21.15.46:0/3693987265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a40328f0 con 0x7ff3a8107b10 2024-09-12T02:40:27.234 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.992+0000 7ff3ad22d640 1 -- 172.21.15.46:0/3693987265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3a4030740 con 0x7ff3a8107b10 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 -- 172.21.15.46:0/3693987265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 msgr2=0x7ff3a8107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 --2- 172.21.15.46:0/3693987265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009820 tx=0x7ff3a402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 -- 172.21.15.46:0/3693987265 shutdown_connections 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 --2- 172.21.15.46:0/3693987265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a8107f10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 -- 172.21.15.46:0/3693987265 >> 172.21.15.46:0/3693987265 conn(0x7ff3a8101460 msgr2=0x7ff3a8103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 -- 172.21.15.46:0/3693987265 shutdown_connections 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.993+0000 7ff3b0493640 1 -- 172.21.15.46:0/3693987265 wait complete. 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3b0493640 1 Processor -- start 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3b0493640 1 -- start start 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3b0493640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3b0493640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8106510 con 0x7ff3a8107b10 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3ae22f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3ae22f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56820/0 (socket says 172.21.15.46:56820) 2024-09-12T02:40:27.235 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3ae22f640 1 -- 172.21.15.46:0/4292006711 learned_addr learned my addr 172.21.15.46:0/4292006711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.994+0000 7ff3ae22f640 1 -- 172.21.15.46:0/4292006711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a819dc50 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff3ae22f640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4030cc0 tx=0x7ff3a4030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3a4042070 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a4030e60 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a819dee0 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3a4037040 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.995+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a81a0bd0 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff3a4045070 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff39f7fe640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 0x7ff39403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8107f90 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff3a406bfd0 con 0x7ff3a8107b10 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff3ada2e640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 0x7ff39403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:27.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:25.996+0000 7ff3ada2e640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 0x7ff39403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff3980098c0 tx=0x7ff398006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.001+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3a404b170 con 0x7ff3a8107b10 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.196+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff3a8062630 con 0x7ff394038fc0 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.201+0000 7ff39f7fe640 1 -- 172.21.15.46:0/4292006711 <== mgr.14118 v2:172.21.15.46:6800/4279295285 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff3a8062630 con 0x7ff394038fc0 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 msgr2=0x7ff39403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 0x7ff39403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff3980098c0 tx=0x7ff398006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 msgr2=0x7ff3a819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4030cc0 tx=0x7ff3a4030cf0 comp rx=0 tx=0).stop 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 shutdown_connections 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff394038fc0 0x7ff39403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 --2- 172.21.15.46:0/4292006711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a8107b10 0x7ff3a819d710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 >> 172.21.15.46:0/4292006711 conn(0x7ff3a8101460 msgr2=0x7ff3a8193dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 shutdown_connections 2024-09-12T02:40:27.237 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:26.203+0000 7ff3b0493640 1 -- 172.21.15.46:0/4292006711 wait complete. 2024-09-12T02:40:27.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:27 smithi046 ceph-mon[30951]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:27.454 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:27 smithi046 ceph-mon[30951]: Saving service mgr spec with placement count:2 2024-09-12T02:40:27.454 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:27 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:28.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f320c228640 1 Processor -- start 2024-09-12T02:40:28.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f320c228640 1 -- start start 2024-09-12T02:40:28.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f320c228640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:28.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f320c228640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f320407c8c0 con 0x7f320407de90 2024-09-12T02:40:28.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f3209fc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f3209fc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56828/0 (socket says 172.21.15.46:56828) 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.637+0000 7f3209fc4640 1 -- 172.21.15.46:0/4024885876 learned_addr learned my addr 172.21.15.46:0/4024885876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.638+0000 7f3209fc4640 1 -- 172.21.15.46:0/4024885876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320407ca40 con 0x7f320407de90 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.638+0000 7f3209fc4640 1 --2- 172.21.15.46:0/4024885876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3200009820 tx=0x7f320002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d77f79ec87b54d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f3208fc2640 1 -- 172.21.15.46:0/4024885876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3200037070 con 0x7f320407de90 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f3208fc2640 1 -- 172.21.15.46:0/4024885876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32000328f0 con 0x7f320407de90 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f3208fc2640 1 -- 172.21.15.46:0/4024885876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3200030740 con 0x7f320407de90 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f320c228640 1 -- 172.21.15.46:0/4024885876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 msgr2=0x7f320407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f320c228640 1 --2- 172.21.15.46:0/4024885876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3200009820 tx=0x7f320002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f320c228640 1 -- 172.21.15.46:0/4024885876 shutdown_connections 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f320c228640 1 --2- 172.21.15.46:0/4024885876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320407c2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.639+0000 7f320c228640 1 -- 172.21.15.46:0/4024885876 >> 172.21.15.46:0/4024885876 conn(0x7f3204102d40 msgr2=0x7f3204105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:28.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 -- 172.21.15.46:0/4024885876 shutdown_connections 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 -- 172.21.15.46:0/4024885876 wait complete. 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 Processor -- start 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 -- start start 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.640+0000 7f320c228640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f320410dbc0 con 0x7f320407de90 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f3209fc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f3209fc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56840/0 (socket says 172.21.15.46:56840) 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f3209fc4640 1 -- 172.21.15.46:0/2240970911 learned_addr learned my addr 172.21.15.46:0/2240970911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f3209fc4640 1 -- 172.21.15.46:0/2240970911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419dca0 con 0x7f320407de90 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f3209fc4640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3200030cc0 tx=0x7f3200030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3200042070 con 0x7f320407de90 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3200030e60 con 0x7f320407de90 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419df30 con 0x7f320407de90 2024-09-12T02:40:28.466 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3200037040 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.641+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a0c20 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.642+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3200045070 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.642+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320407c370 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.642+0000 7f31faffd640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 0x7f31f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.642+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f320006c060 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.643+0000 7f32097c3640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 0x7f31f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.643+0000 7f32097c3640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 0x7f31f003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31f40098c0 tx=0x7f31f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.647+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3200046e00 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.845+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f32041a1210 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.848+0000 7f31faffd640 1 -- 172.21.15.46:0/2240970911 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3200045360 con 0x7f320407de90 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 msgr2=0x7f31f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 0x7f31f003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31f40098c0 tx=0x7f31f4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 msgr2=0x7f320419d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:28.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3200030cc0 tx=0x7f3200030cf0 comp rx=0 tx=0).stop 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 shutdown_connections 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f31f0038fc0 0x7f31f003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 --2- 172.21.15.46:0/2240970911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320407de90 0x7f320419d760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 >> 172.21.15.46:0/2240970911 conn(0x7f3204102d40 msgr2=0x7f3204103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 shutdown_connections 2024-09-12T02:40:28.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:27.850+0000 7f320c228640 1 -- 172.21.15.46:0/2240970911 wait complete. 2024-09-12T02:40:29.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:28 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:29.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:28 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2240970911' entity='client.admin' 2024-09-12T02:40:29.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:28 smithi046 ceph-mon[30951]: from='mgr.14118 172.21.15.46:0/3518619429' entity='mgr.x' 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.902+0000 7f81c09e5640 1 Processor -- start 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.902+0000 7f81c09e5640 1 -- start start 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.902+0000 7f81c09e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.902+0000 7f81c09e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc107f60 con 0x7f81bc107590 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.903+0000 7f81ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.903+0000 7f81ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56856/0 (socket says 172.21.15.46:56856) 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.903+0000 7f81ba59c640 1 -- 172.21.15.46:0/962521138 learned_addr learned my addr 172.21.15.46:0/962521138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.903+0000 7f81ba59c640 1 -- 172.21.15.46:0/962521138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc108790 con 0x7f81bc107590 2024-09-12T02:40:30.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81ba59c640 1 --2- 172.21.15.46:0/962521138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009820 tx=0x7f81ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f375e180b27bfe28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81b959a640 1 -- 172.21.15.46:0/962521138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81bc107590 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81b959a640 1 -- 172.21.15.46:0/962521138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81ac0328f0 con 0x7f81bc107590 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81b959a640 1 -- 172.21.15.46:0/962521138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ac030740 con 0x7f81bc107590 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81c09e5640 1 -- 172.21.15.46:0/962521138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 msgr2=0x7f81bc107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.904+0000 7f81c09e5640 1 --2- 172.21.15.46:0/962521138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009820 tx=0x7f81ac02a270 comp rx=0 tx=0).stop 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 -- 172.21.15.46:0/962521138 shutdown_connections 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 --2- 172.21.15.46:0/962521138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc107990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 -- 172.21.15.46:0/962521138 >> 172.21.15.46:0/962521138 conn(0x7f81bc102d40 msgr2=0x7f81bc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 -- 172.21.15.46:0/962521138 shutdown_connections 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 -- 172.21.15.46:0/962521138 wait complete. 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 Processor -- start 2024-09-12T02:40:30.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.905+0000 7f81c09e5640 1 -- start start 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81c09e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81c09e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc10bb40 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81ba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56862/0 (socket says 172.21.15.46:56862) 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81ba59c640 1 -- 172.21.15.46:0/41973714 learned_addr learned my addr 172.21.15.46:0/41973714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81ba59c640 1 -- 172.21.15.46:0/41973714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc07e0c0 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81ba59c640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097f0 tx=0x7f81ac030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ac042070 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81ac030d80 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc07c2b0 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ac037040 con 0x7f81bc107590 2024-09-12T02:40:30.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.906+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc07c7f0 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.907+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f81ac045070 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.907+0000 7f81b37fe640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 0x7f81a803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.907+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc10bcc0 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.908+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81ac06c120 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.908+0000 7f81b9d9b640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 0x7f81a803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.908+0000 7f81b9d9b640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 0x7f81a803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81a40098c0 tx=0x7f81a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:28.912+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81ac045360 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.092+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f81bc107a10 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.094+0000 7f81b37fe640 1 -- 172.21.15.46:0/41973714 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f81ac0500c0 con 0x7f81bc107590 2024-09-12T02:40:30.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.096+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 msgr2=0x7f81a803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.096+0000 7f81c09e5640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 0x7f81a803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81a40098c0 tx=0x7f81a4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.096+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 msgr2=0x7f81bc07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.096+0000 7f81c09e5640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097f0 tx=0x7f81ac030c70 comp rx=0 tx=0).stop 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 shutdown_connections 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f81a8038f70 0x7f81a803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 --2- 172.21.15.46:0/41973714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc107590 0x7f81bc07db80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 >> 172.21.15.46:0/41973714 conn(0x7f81bc102d40 msgr2=0x7f81bc1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 shutdown_connections 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:29.097+0000 7f81c09e5640 1 -- 172.21.15.46:0/41973714 wait complete. 2024-09-12T02:40:30.146 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-09-12T02:40:30.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:30 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/41973714' entity='client.admin' 2024-09-12T02:40:31.459 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3794585628' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T02:40:31.459 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setuser ceph since I am not root 2024-09-12T02:40:31.459 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setgroup ceph since I am not root 2024-09-12T02:40:31.459 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.255+0000 7f0b18390040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:40:31.459 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.314+0000 7f0b18390040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:40:31.734 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.471+0000 7f0b18390040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:40:31.734 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.540+0000 7f0b18390040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:40:31.734 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.694+0000 7f0b18390040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:40:31.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.600+0000 7f5cbd66a640 1 Processor -- start 2024-09-12T02:40:31.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.600+0000 7f5cbd66a640 1 -- start start 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.600+0000 7f5cbd66a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.600+0000 7f5cbd66a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb807be40 con 0x7f5cb81080b0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.601+0000 7f5cb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.601+0000 7f5cb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56876/0 (socket says 172.21.15.46:56876) 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.601+0000 7f5cb6ffd640 1 -- 172.21.15.46:0/1796087404 learned_addr learned my addr 172.21.15.46:0/1796087404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.601+0000 7f5cb6ffd640 1 -- 172.21.15.46:0/1796087404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb810a9e0 con 0x7f5cb81080b0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.602+0000 7f5cb6ffd640 1 --2- 172.21.15.46:0/1796087404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009820 tx=0x7f5cb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfa8520728c790c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.602+0000 7f5cb5ffb640 1 -- 172.21.15.46:0/1796087404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb0037070 con 0x7f5cb81080b0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.602+0000 7f5cb5ffb640 1 -- 172.21.15.46:0/1796087404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb00328f0 con 0x7f5cb81080b0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.602+0000 7f5cb5ffb640 1 -- 172.21.15.46:0/1796087404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb0030740 con 0x7f5cb81080b0 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 -- 172.21.15.46:0/1796087404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 msgr2=0x7f5cb810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:31.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/1796087404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009820 tx=0x7f5cb002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 -- 172.21.15.46:0/1796087404 shutdown_connections 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/1796087404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb810a4a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 -- 172.21.15.46:0/1796087404 >> 172.21.15.46:0/1796087404 conn(0x7f5cb8101c50 msgr2=0x7f5cb8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 -- 172.21.15.46:0/1796087404 shutdown_connections 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 -- 172.21.15.46:0/1796087404 wait complete. 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.603+0000 7f5cbd66a640 1 Processor -- start 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cbd66a640 1 -- start start 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cbd66a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cbd66a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb810b930 con 0x7f5cb81080b0 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56880/0 (socket says 172.21.15.46:56880) 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cb6ffd640 1 -- 172.21.15.46:0/3794585628 learned_addr learned my addr 172.21.15.46:0/3794585628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cb6ffd640 1 -- 172.21.15.46:0/3794585628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb819ba20 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.604+0000 7f5cb6ffd640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0030cc0 tx=0x7f5cb0030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.605+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb0042070 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.605+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb819bcb0 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.605+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb0030e60 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.605+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb0037040 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.605+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb819e9a0 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.606+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5cb0045070 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.606+0000 7f5caffff640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 0x7f5ca403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.606+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb8108510 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.606+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cb006c060 con 0x7f5cb81080b0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.606+0000 7f5cb67fc640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 0x7f5ca403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:31.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.607+0000 7f5cb67fc640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 0x7f5ca403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00098c0 tx=0x7f5ca0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.611+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cb0045360 con 0x7f5cb81080b0 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:30.836+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5cb819ef80 con 0x7f5cb81080b0 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.097+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5cb002ed90 con 0x7f5cb81080b0 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.098+0000 7f5caffff640 1 -- 172.21.15.46:0/3794585628 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5cb003c740 con 0x7f5cb81080b0 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.099+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 msgr2=0x7f5ca403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 0x7f5ca403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00098c0 tx=0x7f5ca0006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 msgr2=0x7f5cb819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0030cc0 tx=0x7f5cb0030cf0 comp rx=0 tx=0).stop 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 shutdown_connections 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7f5ca4038f70 0x7f5ca403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 --2- 172.21.15.46:0/3794585628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cb81080b0 0x7f5cb819b4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:31.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 >> 172.21.15.46:0/3794585628 conn(0x7f5cb8101c50 msgr2=0x7f5cb8102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:31.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 shutdown_connections 2024-09-12T02:40:31.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:31.100+0000 7f5cbd66a640 1 -- 172.21.15.46:0/3794585628 wait complete. 2024-09-12T02:40:31.988 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:31.845+0000 7f0b18390040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:40:32.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:32 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3794585628' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T02:40:32.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:32 smithi046 ceph-mon[30951]: mgrmap e9: x(active, since 19s) 2024-09-12T02:40:32.244 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:32 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:32.179+0000 7f0b18390040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:40:32.245 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:32 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:32.243+0000 7f0b18390040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:40:32.532 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:32 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:32.440+0000 7f0b18390040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:40:32.948 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb30d7f5640 1 Processor -- start 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb30d7f5640 1 -- start start 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb30d7f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb30d7f5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb308107f60 con 0x7fb308107590 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb306ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:32.949 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb306ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56902/0 (socket says 172.21.15.46:56902) 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.167+0000 7fb306ffd640 1 -- 172.21.15.46:0/2700323951 learned_addr learned my addr 172.21.15.46:0/2700323951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.168+0000 7fb306ffd640 1 -- 172.21.15.46:0/2700323951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb308108790 con 0x7fb308107590 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.168+0000 7fb306ffd640 1 --2- 172.21.15.46:0/2700323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009820 tx=0x7fb2f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcb7bcde92dfac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb305ffb640 1 -- 172.21.15.46:0/2700323951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f8037070 con 0x7fb308107590 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb305ffb640 1 -- 172.21.15.46:0/2700323951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f80328f0 con 0x7fb308107590 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb305ffb640 1 -- 172.21.15.46:0/2700323951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f8030740 con 0x7fb308107590 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 -- 172.21.15.46:0/2700323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 msgr2=0x7fb308107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/2700323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009820 tx=0x7fb2f802a270 comp rx=0 tx=0).stop 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 -- 172.21.15.46:0/2700323951 shutdown_connections 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/2700323951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb308107990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 -- 172.21.15.46:0/2700323951 >> 172.21.15.46:0/2700323951 conn(0x7fb308102d40 msgr2=0x7fb308105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 -- 172.21.15.46:0/2700323951 shutdown_connections 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.169+0000 7fb30d7f5640 1 -- 172.21.15.46:0/2700323951 wait complete. 2024-09-12T02:40:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb30d7f5640 1 Processor -- start 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb30d7f5640 1 -- start start 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb30d7f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb30d7f5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb30810bb40 con 0x7fb308107590 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb306ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb306ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56904/0 (socket says 172.21.15.46:56904) 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.170+0000 7fb306ffd640 1 -- 172.21.15.46:0/1582873392 learned_addr learned my addr 172.21.15.46:0/1582873392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb306ffd640 1 -- 172.21.15.46:0/1582873392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30807e0c0 con 0x7fb308107590 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb306ffd640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009940 tx=0x7fb2f8030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f8042070 con 0x7fb308107590 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f8030e30 con 0x7fb308107590 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30807c2b0 con 0x7fb308107590 2024-09-12T02:40:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f8037040 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.171+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30807c7f0 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb2f8045070 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb308107a10 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb2fffff640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 0x7fb2f403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb2f806c040 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb3067fc640 1 -- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 msgr2=0x7fb2f403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.172+0000 7fb3067fc640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 0x7fb2f403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.177+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2f804b170 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.373+0000 7fb3067fc640 1 -- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 msgr2=0x7fb2f403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.373+0000 7fb3067fc640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 0x7fb2f403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.402+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb30807d100 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.402+0000 7fb2fffff640 1 -- 172.21.15.46:0/1582873392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb2f802fd90 con 0x7fb308107590 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 msgr2=0x7fb2f403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:40:32.952 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 0x7fb2f403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 msgr2=0x7fb30807db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009940 tx=0x7fb2f8030cc0 comp rx=0 tx=0).stop 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 shutdown_connections 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7fb2f4039020 0x7fb2f403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 --2- 172.21.15.46:0/1582873392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb308107590 0x7fb30807db80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 >> 172.21.15.46:0/1582873392 conn(0x7fb308102d40 msgr2=0x7fb3081038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 shutdown_connections 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:32.404+0000 7fb30d7f5640 1 -- 172.21.15.46:0/1582873392 wait complete. 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-12T02:40:32.953 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-09-12T02:40:33.327 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1582873392' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:40:33.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:33 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:33.383+0000 7f0b18390040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:40:33.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:33 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:33.453+0000 7f0b18390040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:40:34.282 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.198+0000 7f0b18390040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:40:34.689 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.318+0000 7f0b18390040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:40:34.689 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.468+0000 7f0b18390040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:40:34.689 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.531+0000 7f0b18390040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:40:34.689 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.687+0000 7f0b18390040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:40:34.986 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.757+0000 7f0b18390040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:40:34.986 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.820+0000 7f0b18390040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:40:34.986 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:34.985+0000 7f0b18390040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:40:35.271 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:35.053+0000 7f0b18390040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:40:35.271 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:35.122+0000 7f0b18390040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:40:35.271 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:35.208+0000 7f0b18390040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:40:35.271 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:35.270+0000 7f0b18390040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:40:35.532 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:35.332+0000 7f0b18390040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:40:36.406 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:36.155+0000 7f0b18390040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: Active manager daemon x restarted 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: Activating manager daemon x 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: mgrmap e10: x(active, starting, since 0.00341437s) 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:40:36.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: Manager daemon x is now available 2024-09-12T02:40:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:40:36.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff067237640 1 Processor -- start 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff067237640 1 -- start start 2024-09-12T02:40:38.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff067237640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff067237640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff060075a20 con 0x7ff060108070 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff064fd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff064fd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56912/0 (socket says 172.21.15.46:56912) 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff064fd3640 1 -- 172.21.15.46:0/1614364151 learned_addr learned my addr 172.21.15.46:0/1614364151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.350+0000 7ff064fd3640 1 -- 172.21.15.46:0/1614364151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06010a9a0 con 0x7ff060108070 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff064fd3640 1 --2- 172.21.15.46:0/1614364151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff054009820 tx=0x7ff05402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909252d6aab21557 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff05f7fe640 1 -- 172.21.15.46:0/1614364151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054037070 con 0x7ff060108070 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff05f7fe640 1 -- 172.21.15.46:0/1614364151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0540328f0 con 0x7ff060108070 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff05f7fe640 1 -- 172.21.15.46:0/1614364151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054030740 con 0x7ff060108070 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff067237640 1 -- 172.21.15.46:0/1614364151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 msgr2=0x7ff06010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.352+0000 7ff067237640 1 --2- 172.21.15.46:0/1614364151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff054009820 tx=0x7ff05402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- 172.21.15.46:0/1614364151 shutdown_connections 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 --2- 172.21.15.46:0/1614364151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06010a460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- 172.21.15.46:0/1614364151 >> 172.21.15.46:0/1614364151 conn(0x7ff060101c30 msgr2=0x7ff060104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- 172.21.15.46:0/1614364151 shutdown_connections 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- 172.21.15.46:0/1614364151 wait complete. 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 Processor -- start 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- start start 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.353+0000 7ff067237640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff06010b9c0 con 0x7ff060108070 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff064fd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff064fd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56920/0 (socket says 172.21.15.46:56920) 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff064fd3640 1 -- 172.21.15.46:0/4177404657 learned_addr learned my addr 172.21.15.46:0/4177404657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff064fd3640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06019dc60 con 0x7ff060108070 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff064fd3640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff054030cc0 tx=0x7ff054030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054042070 con 0x7ff060108070 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff054030e60 con 0x7ff060108070 2024-09-12T02:40:38.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06019def0 con 0x7ff060108070 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff054037040 con 0x7ff060108070 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.354+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a0be0 con 0x7ff060108070 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff054045070 con 0x7ff060108070 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05dffb640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff05003bbf0 con 0x7ff050039020 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff05406c280 con 0x7ff060108070 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05ffff640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 msgr2=0x7ff05003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.356+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.557+0000 7ff05ffff640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 msgr2=0x7ff05003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.557+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.958+0000 7ff05ffff640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 msgr2=0x7ff05003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:33.958+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:34.759+0000 7ff05ffff640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 msgr2=0x7ff05003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4279295285 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:34.759+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:40:38.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:36.159+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7ff05406ad30 con 0x7ff060108070 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:36.159+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 msgr2=0x7ff05003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:36.159+0000 7ff05dffb640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.161+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ff05403c630 con 0x7ff060108070 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.161+0000 7ff05dffb640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 0x7ff05003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.161+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff05402eec0 con 0x7ff05003cae0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.162+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 0x7ff05003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.162+0000 7ff05ffff640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 0x7ff05003eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff04c003980 tx=0x7ff04c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.163+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff05402eec0 con 0x7ff05003cae0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.170+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff0601084d0 con 0x7ff05003cae0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.170+0000 7ff05dffb640 1 -- 172.21.15.46:0/4177404657 <== mgr.14150 v2:172.21.15.46:6800/3748867549 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff0601084d0 con 0x7ff05003cae0 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 msgr2=0x7ff05003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 0x7ff05003eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff04c003980 tx=0x7ff04c0092b0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 msgr2=0x7ff06019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:38.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff054030cc0 tx=0x7ff054030cf0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 shutdown_connections 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff05003cae0 0x7ff05003eed0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:6800/4279295285,v1:172.21.15.46:6801/4279295285] conn(0x7ff050039020 0x7ff05003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 --2- 172.21.15.46:0/4177404657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff060108070 0x7ff06019d720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 >> 172.21.15.46:0/4177404657 conn(0x7ff060101c30 msgr2=0x7ff060102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 shutdown_connections 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:37.171+0000 7ff067237640 1 -- 172.21.15.46:0/4177404657 wait complete. 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-09-12T02:40:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-09-12T02:40:38.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-mon[30951]: mgrmap e11: x(active, since 1.00668s) 2024-09-12T02:40:38.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:38.743 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:38.655+0000 7f0a83664640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:40:38] ENGINE Error in HTTPServer.serve 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Traceback (most recent call last): 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._connections.run(self.expiration_interval) 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._run(expiration_interval) 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s = self.context.wrap_socket( 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:40:38.744 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: return self.sslsocket_class._create( 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self.do_handshake() 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._sslobj.do_handshake() 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:40:38.745 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:40:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:40:39.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T02:40:39.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16af282640 1 Processor -- start 2024-09-12T02:40:39.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16af282640 1 -- start start 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16af282640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16af282640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8075a40 con 0x7f16a8108060 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16ad01e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16ad01e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36444/0 (socket says 172.21.15.46:36444) 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.549+0000 7f16ad01e640 1 -- 172.21.15.46:0/2188277694 learned_addr learned my addr 172.21.15.46:0/2188277694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.550+0000 7f16ad01e640 1 -- 172.21.15.46:0/2188277694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a810a990 con 0x7f16a8108060 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.551+0000 7f16ad01e640 1 --2- 172.21.15.46:0/2188277694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009820 tx=0x7f16a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b45304389d0bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.551+0000 7f169ffff640 1 -- 172.21.15.46:0/2188277694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16a4037070 con 0x7f16a8108060 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.551+0000 7f169ffff640 1 -- 172.21.15.46:0/2188277694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a40328f0 con 0x7f16a8108060 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.551+0000 7f169ffff640 1 -- 172.21.15.46:0/2188277694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16a4030740 con 0x7f16a8108060 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 -- 172.21.15.46:0/2188277694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 msgr2=0x7f16a810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 --2- 172.21.15.46:0/2188277694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009820 tx=0x7f16a402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 -- 172.21.15.46:0/2188277694 shutdown_connections 2024-09-12T02:40:39.332 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 --2- 172.21.15.46:0/2188277694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a810a450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 -- 172.21.15.46:0/2188277694 >> 172.21.15.46:0/2188277694 conn(0x7f16a8101c60 msgr2=0x7f16a8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 -- 172.21.15.46:0/2188277694 shutdown_connections 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.552+0000 7f16af282640 1 -- 172.21.15.46:0/2188277694 wait complete. 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16af282640 1 Processor -- start 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16af282640 1 -- start start 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16af282640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16af282640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16a810b9b0 con 0x7f16a8108060 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16ad01e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16ad01e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36456/0 (socket says 172.21.15.46:36456) 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16ad01e640 1 -- 172.21.15.46:0/3731976708 learned_addr learned my addr 172.21.15.46:0/3731976708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.553+0000 7f16ad01e640 1 -- 172.21.15.46:0/3731976708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a819ba40 con 0x7f16a8108060 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f16ad01e640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097f0 tx=0x7f16a4030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:39.333 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16a4042070 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a4030d80 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a819bcd0 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16a4037040 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.554+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a819e9c0 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.555+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f16a4045070 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.555+0000 7f169e7fc640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 0x7f169403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.555+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f16a406c370 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.555+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a81084c0 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.555+0000 7f16ac81d640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 0x7f169403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.556+0000 7f16ac81d640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 0x7f169403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16980098c0 tx=0x7f1698006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.560+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16a403c730 con 0x7f16a8108060 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.742+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f16a8102ec0 con 0x7f1694038fc0 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.797+0000 7f169e7fc640 1 -- 172.21.15.46:0/3731976708 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f16a8102ec0 con 0x7f1694038fc0 2024-09-12T02:40:39.334 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 msgr2=0x7f169403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 0x7f169403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16980098c0 tx=0x7f1698006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 msgr2=0x7f16a819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097f0 tx=0x7f16a4030c70 comp rx=0 tx=0).stop 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 shutdown_connections 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1694038fc0 0x7f169403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 --2- 172.21.15.46:0/3731976708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16a8108060 0x7f16a819b500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 >> 172.21.15.46:0/3731976708 conn(0x7f16a8101c60 msgr2=0x7f16a81027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 shutdown_connections 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:38.800+0000 7f16af282640 1 -- 172.21.15.46:0/3731976708 wait complete. 2024-09-12T02:40:39.335 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: mgrmap e12: x(active, since 2s) 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:39.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:38] ENGINE Bus STARTING 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:38] ENGINE Serving on https://172.21.15.46:7150 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:38] ENGINE Error in HTTPServer.serve 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:40:40.426 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:40:40.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:40.428 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:38] ENGINE Serving on http://172.21.15.46:8765 2024-09-12T02:40:40.428 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:40 smithi046 ceph-mon[30951]: [12/Sep/2024:02:40:38] ENGINE Bus STARTED 2024-09-12T02:40:41.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:41 smithi046 ceph-mon[30951]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:41.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:41.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$HseruZeS4SrUMvcXOJoDvuapV6q8Foklt8iFxo2jS/iKfBF7l2Sba", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726108840, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f3a752640 1 Processor -- start 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f3a752640 1 -- start start 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f3a752640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f3a752640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3407be00 con 0x7f6f341080f0 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36468/0 (socket says 172.21.15.46:36468) 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f33fff640 1 -- 172.21.15.46:0/4282538616 learned_addr learned my addr 172.21.15.46:0/4282538616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:41.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.779+0000 7f6f33fff640 1 -- 172.21.15.46:0/4282538616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3410aa20 con 0x7f6f341080f0 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.780+0000 7f6f33fff640 1 --2- 172.21.15.46:0/4282538616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009820 tx=0x7f6f2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad6cc85ace6c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.781+0000 7f6f337fe640 1 -- 172.21.15.46:0/4282538616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f24037070 con 0x7f6f341080f0 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.781+0000 7f6f337fe640 1 -- 172.21.15.46:0/4282538616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f240328f0 con 0x7f6f341080f0 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.781+0000 7f6f337fe640 1 -- 172.21.15.46:0/4282538616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f24030740 con 0x7f6f341080f0 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.781+0000 7f6f3a752640 1 -- 172.21.15.46:0/4282538616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 msgr2=0x7f6f3410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.781+0000 7f6f3a752640 1 --2- 172.21.15.46:0/4282538616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009820 tx=0x7f6f2402a270 comp rx=0 tx=0).stop 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- 172.21.15.46:0/4282538616 shutdown_connections 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 --2- 172.21.15.46:0/4282538616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f3410a4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- 172.21.15.46:0/4282538616 >> 172.21.15.46:0/4282538616 conn(0x7f6f34101c50 msgr2=0x7f6f34104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- 172.21.15.46:0/4282538616 shutdown_connections 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- 172.21.15.46:0/4282538616 wait complete. 2024-09-12T02:40:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 Processor -- start 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- start start 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f3a752640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410b970 con 0x7f6f341080f0 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36482/0 (socket says 172.21.15.46:36482) 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.782+0000 7f6f33fff640 1 -- 172.21.15.46:0/1525806678 learned_addr learned my addr 172.21.15.46:0/1525806678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f33fff640 1 -- 172.21.15.46:0/1525806678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f34107eb0 con 0x7f6f341080f0 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f33fff640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097f0 tx=0x7f6f24030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f24042070 con 0x7f6f341080f0 2024-09-12T02:40:41.512 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f24030d80 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341060a0 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f24037040 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.783+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341065e0 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.784+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6f24045070 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.785+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f34108550 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.785+0000 7f6f31ffb640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 0x7f6f2003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.785+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f2406c2f0 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.785+0000 7f6f2bfff640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 0x7f6f2003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.785+0000 7f6f2bfff640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 0x7f6f2003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0098c0 tx=0x7f6f1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.790+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f2403add0 con 0x7f6f341080f0 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:39.985+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6f34106860 con 0x7f6f20039020 2024-09-12T02:40:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.235+0000 7f6f31ffb640 1 -- 172.21.15.46:0/1525806678 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6f34106860 con 0x7f6f20039020 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 msgr2=0x7f6f2003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 0x7f6f2003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0098c0 tx=0x7f6f1c006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 msgr2=0x7f6f34107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097f0 tx=0x7f6f24030c70 comp rx=0 tx=0).stop 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 shutdown_connections 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6f20039020 0x7f6f2003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 --2- 172.21.15.46:0/1525806678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f341080f0 0x7f6f34107970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 >> 172.21.15.46:0/1525806678 conn(0x7f6f34101c50 msgr2=0x7f6f34102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 shutdown_connections 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:40.237+0000 7f6f3a752640 1 -- 172.21.15.46:0/1525806678 wait complete. 2024-09-12T02:40:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-09-12T02:40:42.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:42 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/350733684' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.883+0000 7f60aea49640 1 Processor -- start 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.883+0000 7f60aea49640 1 -- start start 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.883+0000 7f60aea49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.883+0000 7f60aea49640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8106e80 con 0x7f60a81064b0 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.884+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.884+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36498/0 (socket says 172.21.15.46:36498) 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.884+0000 7f60a7fff640 1 -- 172.21.15.46:0/1146311569 learned_addr learned my addr 172.21.15.46:0/1146311569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:42.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.884+0000 7f60a7fff640 1 -- 172.21.15.46:0/1146311569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81076b0 con 0x7f60a81064b0 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60a7fff640 1 --2- 172.21.15.46:0/1146311569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6098009820 tx=0x7f609802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d13f3899d98721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60a6ffd640 1 -- 172.21.15.46:0/1146311569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098037070 con 0x7f60a81064b0 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60a6ffd640 1 -- 172.21.15.46:0/1146311569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60980328f0 con 0x7f60a81064b0 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60a6ffd640 1 -- 172.21.15.46:0/1146311569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098030740 con 0x7f60a81064b0 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60aea49640 1 -- 172.21.15.46:0/1146311569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 msgr2=0x7f60a81068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60aea49640 1 --2- 172.21.15.46:0/1146311569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6098009820 tx=0x7f609802a270 comp rx=0 tx=0).stop 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60aea49640 1 -- 172.21.15.46:0/1146311569 shutdown_connections 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60aea49640 1 --2- 172.21.15.46:0/1146311569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a81068b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.885+0000 7f60aea49640 1 -- 172.21.15.46:0/1146311569 >> 172.21.15.46:0/1146311569 conn(0x7f60a8101c60 msgr2=0x7f60a8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.886+0000 7f60aea49640 1 -- 172.21.15.46:0/1146311569 shutdown_connections 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.886+0000 7f60aea49640 1 -- 172.21.15.46:0/1146311569 wait complete. 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.886+0000 7f60aea49640 1 Processor -- start 2024-09-12T02:40:42.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.886+0000 7f60aea49640 1 -- start start 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60aea49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60aea49640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810aa60 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36502/0 (socket says 172.21.15.46:36502) 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a7fff640 1 -- 172.21.15.46:0/350733684 learned_addr learned my addr 172.21.15.46:0/350733684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a7fff640 1 -- 172.21.15.46:0/350733684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a819baa0 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a7fff640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6098009940 tx=0x7f6098030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098042070 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6098030e30 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a819bd30 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098037040 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.887+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a819ea20 con 0x7f60a81064b0 2024-09-12T02:40:42.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.888+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6098045070 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.888+0000 7f60a57fa640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 0x7f609003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.888+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a810abe0 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.888+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f609806c0d0 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.888+0000 7f60a77fe640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 0x7f609003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.889+0000 7f60a77fe640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 0x7f609003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098c0 tx=0x7f609c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:41.893+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6098042210 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.067+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f60a802d9b0 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.068+0000 7f60a57fa640 1 -- 172.21.15.46:0/350733684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f609803ae30 con 0x7f60a81064b0 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 msgr2=0x7f609003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 0x7f609003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098c0 tx=0x7f609c006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 msgr2=0x7f60a819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6098009940 tx=0x7f6098030cc0 comp rx=0 tx=0).stop 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 shutdown_connections 2024-09-12T02:40:42.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6090039020 0x7f609003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 --2- 172.21.15.46:0/350733684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60a81064b0 0x7f60a819b560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 >> 172.21.15.46:0/350733684 conn(0x7f60a8101c60 msgr2=0x7f60a8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 shutdown_connections 2024-09-12T02:40:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:42.070+0000 7f60aea49640 1 -- 172.21.15.46:0/350733684 wait complete. 2024-09-12T02:40:42.660 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:40:42.660 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-12T02:40:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:40:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-12T02:40:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-12T02:40:42.669 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T02:40:42.671 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout: Password: 9vsnpswf8n 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:42.672 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config directory 2024-09-12T02:40:43.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:43 smithi046 ceph-mon[30951]: mgrmap e13: x(active, since 6s) 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.067+0000 7f14c5163640 1 Processor -- start 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.067+0000 7f14c5163640 1 -- start start 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.067+0000 7f14c5163640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.067+0000 7f14c5163640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14c007be40 con 0x7f14c0108070 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.068+0000 7f14bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.068+0000 7f14bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36506/0 (socket says 172.21.15.46:36506) 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.068+0000 7f14bed9d640 1 -- 172.21.15.46:0/2839028118 learned_addr learned my addr 172.21.15.46:0/2839028118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.068+0000 7f14bed9d640 1 -- 172.21.15.46:0/2839028118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c010a9a0 con 0x7f14c0108070 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14bed9d640 1 --2- 172.21.15.46:0/2839028118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009820 tx=0x7f14b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ae19e4a794b592 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14bdd9b640 1 -- 172.21.15.46:0/2839028118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0037070 con 0x7f14c0108070 2024-09-12T02:40:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14bdd9b640 1 -- 172.21.15.46:0/2839028118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14b00328f0 con 0x7f14c0108070 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14bdd9b640 1 -- 172.21.15.46:0/2839028118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0030740 con 0x7f14c0108070 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14c5163640 1 -- 172.21.15.46:0/2839028118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 msgr2=0x7f14c010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.069+0000 7f14c5163640 1 --2- 172.21.15.46:0/2839028118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009820 tx=0x7f14b002a270 comp rx=0 tx=0).stop 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 -- 172.21.15.46:0/2839028118 shutdown_connections 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 --2- 172.21.15.46:0/2839028118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c010a460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 -- 172.21.15.46:0/2839028118 >> 172.21.15.46:0/2839028118 conn(0x7f14c0101bf0 msgr2=0x7f14c0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 -- 172.21.15.46:0/2839028118 shutdown_connections 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 -- 172.21.15.46:0/2839028118 wait complete. 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.070+0000 7f14c5163640 1 Processor -- start 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14c5163640 1 -- start start 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14c5163640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14c5163640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14c010b960 con 0x7f14c0108070 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36508/0 (socket says 172.21.15.46:36508) 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14bed9d640 1 -- 172.21.15.46:0/3243753160 learned_addr learned my addr 172.21.15.46:0/3243753160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.071+0000 7f14bed9d640 1 -- 172.21.15.46:0/3243753160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c019ba50 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14bed9d640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14b0030cc0 tx=0x7f14b0030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0042070 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c019bce0 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14b0030e60 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0037040 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.072+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c019e9d0 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.073+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c01084d0 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.073+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f14b0045070 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.074+0000 7f14a7fff640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 0x7f14a803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.074+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14b006c230 con 0x7f14c0108070 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.074+0000 7f14be59c640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 0x7f14a803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:43.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.074+0000 7f14be59c640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 0x7f14a803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098c0 tx=0x7f14b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.078+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14b003b330 con 0x7f14c0108070 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.347+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f14c019eba0 con 0x7f14c0108070 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.349+0000 7f14a7fff640 1 -- 172.21.15.46:0/3243753160 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f14b003d950 con 0x7f14c0108070 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 msgr2=0x7f14a803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 0x7f14a803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098c0 tx=0x7f14b4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 msgr2=0x7f14c019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14b0030cc0 tx=0x7f14b0030cf0 comp rx=0 tx=0).stop 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 shutdown_connections 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f14a8039060 0x7f14a803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 --2- 172.21.15.46:0/3243753160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14c0108070 0x7f14c019b510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.351+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 >> 172.21.15.46:0/3243753160 conn(0x7f14c0101bf0 msgr2=0x7f14c0102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:43.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.352+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 shutdown_connections 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-09-12T02:40:43.352+0000 7f14c5163640 1 -- 172.21.15.46:0/3243753160 wait complete. 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-09-12T02:40:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.894 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-09-12T02:40:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T02:40:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:43.894 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-09-12T02:40:43.928 INFO:tasks.cephadm:Fetching config... 2024-09-12T02:40:43.929 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:40:43.929 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T02:40:43.946 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T02:40:43.946 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:40:43.946 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T02:40:44.002 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T02:40:44.002 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:40:44.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T02:40:44.068 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T02:40:44.068 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:40:44.068 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T02:40:44.124 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T02:40:44.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:40:44.200 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:44.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:40:44.266 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:44.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:40:44.339 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXz9RVpys8i7AgpGd102OBTfENgZk97+roULDPyOoUG/dSthujGbeWEwKkh4dGLFh3yVSk3YbfhtjOwpbVBsjLojWDcEekn1kWFA5wzvn4len3DTeBenQ5vruehja41wH39T0cgzhtWKNrHAlfYMuN/E9Ceaa2nXb2D1sdEKERDEAhTeghWechqPHqEqzjAUXtwISdsV/hyiTP+szFqnZ8dJTwkuxUqUCBbPefOLOKw7VX3I7bxOCueblTOW9jFG410m24U2emJs47YPfy0Kn68avMpFt4PaFiaELq9YhciFEGt9OGMSuHzTOBJHe17p79Iggjbc1xics8nJdS3cJB+DfCHK2hPIjcJzp08BZTh8wANtuIS7BPzGEAmVdU93rfElXahnrrMYVCmyMpsNY9eCAJnSKiNtUrTFUaZyLn/6D50fxFqk49uVqmnYdwH71n6t7EhXXWuudPWrlXwDA4WHnU0vQNjqLuoFLJuDwWS2JvW18y+rWes/wMQZ+3JgU= ceph-2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:40:44.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T02:40:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:44 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3243753160' entity='client.admin' 2024-09-12T02:40:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.071+0000 7f4f3970a640 1 -- 172.21.15.46:0/2330719073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 msgr2=0x7f4f34100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.071+0000 7f4f3970a640 1 --2- 172.21.15.46:0/2330719073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f34100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009880 tx=0x7f4f2402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.071+0000 7f4f3970a640 1 -- 172.21.15.46:0/2330719073 shutdown_connections 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.071+0000 7f4f3970a640 1 --2- 172.21.15.46:0/2330719073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f34100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.071+0000 7f4f3970a640 1 -- 172.21.15.46:0/2330719073 >> 172.21.15.46:0/2330719073 conn(0x7f4f340fbcd0 msgr2=0x7f4f340fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.072+0000 7f4f3970a640 1 -- 172.21.15.46:0/2330719073 shutdown_connections 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.072+0000 7f4f3970a640 1 -- 172.21.15.46:0/2330719073 wait complete. 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.072+0000 7f4f3970a640 1 Processor -- start 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f3970a640 1 -- start start 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f3970a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f3970a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341064a0 con 0x7f4f34100540 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50548/0 (socket says 172.21.15.46:50548) 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f32ffd640 1 -- 172.21.15.46:0/3523005960 learned_addr learned my addr 172.21.15.46:0/3523005960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f32ffd640 1 -- 172.21.15.46:0/3523005960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f341999d0 con 0x7f4f34100540 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f32ffd640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005ec0 tx=0x7f4f240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f2403f070 con 0x7f4f34100540 2024-09-12T02:40:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f34199ba0 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f24003aa0 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f24035030 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.073+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f34199fc0 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.074+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4f24042070 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.074+0000 7f4f30ff9640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 0x7f4f2003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.074+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4f2406cd90 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.075+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f34106c20 con 0x7f4f34100540 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.075+0000 7f4f2bfff640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 0x7f4f2003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.075+0000 7f4f2bfff640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 0x7f4f2003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0098c0 tx=0x7f4f1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.079+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f240433b0 con 0x7f4f34100540 2024-09-12T02:40:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.256+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4f341009c0 con 0x7f4f34100540 2024-09-12T02:40:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.258+0000 7f4f30ff9640 1 -- 172.21.15.46:0/3523005960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4f2402f300 con 0x7f4f34100540 2024-09-12T02:40:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 msgr2=0x7f4f2003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 0x7f4f2003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0098c0 tx=0x7f4f1c006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 msgr2=0x7f4f3406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005ec0 tx=0x7f4f240048f0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 shutdown_connections 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4f20039060 0x7f4f2003b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 --2- 172.21.15.46:0/3523005960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f34100540 0x7f4f3406a170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 >> 172.21.15.46:0/3523005960 conn(0x7f4f340fbcd0 msgr2=0x7f4f340fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 shutdown_connections 2024-09-12T02:40:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:45.261+0000 7f4f3970a640 1 -- 172.21.15.46:0/3523005960 wait complete. 2024-09-12T02:40:46.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:46 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3523005960' entity='client.admin' 2024-09-12T02:40:46.694 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T02:40:46.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T02:40:47.487 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:40:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.911+0000 7fe8d7a82640 1 -- 172.21.15.46:0/3042041644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 msgr2=0x7fe8d0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.911+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/3042041644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d0100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009880 tx=0x7fe8cc02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:40:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- 172.21.15.46:0/3042041644 shutdown_connections 2024-09-12T02:40:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/3042041644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d0100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- 172.21.15.46:0/3042041644 >> 172.21.15.46:0/3042041644 conn(0x7fe8d00fbc80 msgr2=0x7fe8d00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- 172.21.15.46:0/3042041644 shutdown_connections 2024-09-12T02:40:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- 172.21.15.46:0/3042041644 wait complete. 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 Processor -- start 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- start start 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.912+0000 7fe8d7a82640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01064d0 con 0x7fe8d0100510 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d581e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d581e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50570/0 (socket says 172.21.15.46:50570) 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d581e640 1 -- 172.21.15.46:0/73267682 learned_addr learned my addr 172.21.15.46:0/73267682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d581e640 1 -- 172.21.15.46:0/73267682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d00fe0a0 con 0x7fe8d0100510 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d581e640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009230 tx=0x7fe8cc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc03f070 con 0x7fe8d0100510 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8cc003aa0 con 0x7fe8d0100510 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d00fc290 con 0x7fe8d0100510 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8cc035030 con 0x7fe8d0100510 2024-09-12T02:40:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.913+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d00fc7d0 con 0x7fe8d0100510 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.914+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8cc042070 con 0x7fe8d0100510 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.914+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0106c50 con 0x7fe8d0100510 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.914+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 0x7fe8bc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.915+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8cc06ce60 con 0x7fe8d0100510 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.915+0000 7fe8d501d640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 0x7fe8bc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.915+0000 7fe8d501d640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 0x7fe8bc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098c0 tx=0x7fe8c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:47.919+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8cc036260 con 0x7fe8d0100510 2024-09-12T02:40:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.099+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe8d0190280 con 0x7fe8bc039060 2024-09-12T02:40:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.103+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/73267682 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8d0190280 con 0x7fe8bc039060 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 msgr2=0x7fe8bc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 0x7fe8bc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098c0 tx=0x7fe8c4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 msgr2=0x7fe8d00fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009230 tx=0x7fe8cc0048f0 comp rx=0 tx=0).stop 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 shutdown_connections 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe8bc039060 0x7fe8bc03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 --2- 172.21.15.46:0/73267682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d0100510 0x7fe8d00fdb60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 >> 172.21.15.46:0/73267682 conn(0x7fe8d00fbc80 msgr2=0x7fe8d006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 shutdown_connections 2024-09-12T02:40:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:48.105+0000 7fe8d7a82640 1 -- 172.21.15.46:0/73267682 wait complete. 2024-09-12T02:40:48.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:40:48.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:48 smithi046 ceph-mon[30951]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-12T02:40:48.808 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-09-12T02:40:48.808 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:40:48.809 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:40:48.827 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:40:48.827 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:40:48.886 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-09-12T02:40:48.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch host add smithi145 2024-09-12T02:40:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 -- 172.21.15.46:0/4237651621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 msgr2=0x7f5b70106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/4237651621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70106940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009880 tx=0x7f5b6002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 -- 172.21.15.46:0/4237651621 shutdown_connections 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/4237651621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70106940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 -- 172.21.15.46:0/4237651621 >> 172.21.15.46:0/4237651621 conn(0x7f5b700fc1b0 msgr2=0x7f5b700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 -- 172.21.15.46:0/4237651621 shutdown_connections 2024-09-12T02:40:49.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.573+0000 7f5b74fa0640 1 -- 172.21.15.46:0/4237651621 wait complete. 2024-09-12T02:40:49.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b74fa0640 1 Processor -- start 2024-09-12T02:40:49.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b74fa0640 1 -- start start 2024-09-12T02:40:49.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b74fa0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:49.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b74fa0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010aa40 con 0x7f5b70106560 2024-09-12T02:40:49.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b6e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b6e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50578/0 (socket says 172.21.15.46:50578) 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b6e59c640 1 -- 172.21.15.46:0/2371470049 learned_addr learned my addr 172.21.15.46:0/2371470049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.574+0000 7f5b6e59c640 1 -- 172.21.15.46:0/2371470049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b70197540 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b6e59c640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009230 tx=0x7f5b600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b6003f070 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b60004d10 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701977d0 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b60035030 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.575+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7019b8d0 con 0x7f5b70106560 2024-09-12T02:40:49.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.576+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5b60042070 con 0x7f5b70106560 2024-09-12T02:40:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.576+0000 7f5b677fe640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 0x7f5b5c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.576+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b6006ccb0 con 0x7f5b70106560 2024-09-12T02:40:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.576+0000 7f5b67fff640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 0x7f5b5c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.576+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70101bd0 con 0x7f5b70106560 2024-09-12T02:40:49.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.577+0000 7f5b67fff640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 0x7f5b5c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b580098c0 tx=0x7f5b58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.581+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b60039350 con 0x7f5b70106560 2024-09-12T02:40:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:49.760+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}) -- 0x7f5b701069c0 con 0x7f5b5c0390c0 2024-09-12T02:40:50.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:40:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:40:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:40:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:50 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:51.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:51 smithi046 ceph-mon[30951]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:52.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:52 smithi046 ceph-mon[30951]: Deploying cephadm binary to smithi145 2024-09-12T02:40:53.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.578+0000 7f5b677fe640 1 -- 172.21.15.46:0/2371470049 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5b701069c0 con 0x7f5b5c0390c0 2024-09-12T02:40:53.579 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-09-12T02:40:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 msgr2=0x7f5b5c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 0x7f5b5c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b580098c0 tx=0x7f5b58006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 msgr2=0x7f5b70197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009230 tx=0x7f5b600048f0 comp rx=0 tx=0).stop 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 shutdown_connections 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5b5c0390c0 0x7f5b5c03b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 --2- 172.21.15.46:0/2371470049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b70106560 0x7f5b70197000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 >> 172.21.15.46:0/2371470049 conn(0x7f5b700fc1b0 msgr2=0x7f5b70069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 shutdown_connections 2024-09-12T02:40:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:53.581+0000 7f5b74fa0640 1 -- 172.21.15.46:0/2371470049 wait complete. 2024-09-12T02:40:54.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:40:54.574 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:40:54.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:54.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:54 smithi046 ceph-mon[30951]: Added host smithi145 2024-09-12T02:40:54.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:40:54.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:54.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 -- 172.21.15.46:0/1441663553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 msgr2=0x7f8200107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 --2- 172.21.15.46:0/1441663553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009880 tx=0x7f81f002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 -- 172.21.15.46:0/1441663553 shutdown_connections 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 --2- 172.21.15.46:0/1441663553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 -- 172.21.15.46:0/1441663553 >> 172.21.15.46:0/1441663553 conn(0x7f82000fd290 msgr2=0x7f82000ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 -- 172.21.15.46:0/1441663553 shutdown_connections 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.013+0000 7f8206d49640 1 -- 172.21.15.46:0/1441663553 wait complete. 2024-09-12T02:40:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.014+0000 7f8206d49640 1 Processor -- start 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.014+0000 7f8206d49640 1 -- start start 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8206d49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8206d49640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f820010bb40 con 0x7f8200107660 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8204ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8204ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48692/0 (socket says 172.21.15.46:48692) 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8204ae5640 1 -- 172.21.15.46:0/1992369682 learned_addr learned my addr 172.21.15.46:0/1992369682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8204ae5640 1 -- 172.21.15.46:0/1992369682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8200197580 con 0x7f8200107660 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8204ae5640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009230 tx=0x7f81f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f003f070 con 0x7f8200107660 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81f0004d10 con 0x7f8200107660 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8200197810 con 0x7f8200107660 2024-09-12T02:40:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.015+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81f0035030 con 0x7f8200107660 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.016+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820019b910 con 0x7f8200107660 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.016+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f81f0042070 con 0x7f8200107660 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.016+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8200102cd0 con 0x7f8200107660 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.017+0000 7f81fdffb640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 0x7f81e803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.017+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f81f006cde0 con 0x7f8200107660 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.017+0000 7f81fffff640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 0x7f81e803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.017+0000 7f81fffff640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 0x7f81e803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81f40098c0 tx=0x7f81f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.021+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81f00037d0 con 0x7f8200107660 2024-09-12T02:40:55.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.229+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82001983a0 con 0x7f81e8039060 2024-09-12T02:40:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.230+0000 7f81fdffb640 1 -- 172.21.15.46:0/1992369682 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f82001983a0 con 0x7f81e8039060 2024-09-12T02:40:55.231 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:40:55.231 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-09-12T02:40:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 msgr2=0x7f81e803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 0x7f81e803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81f40098c0 tx=0x7f81f4006db0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 msgr2=0x7f8200197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009230 tx=0x7f81f00048f0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 shutdown_connections 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f81e8039060 0x7f81e803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 --2- 172.21.15.46:0/1992369682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8200107660 0x7f8200197040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 >> 172.21.15.46:0/1992369682 conn(0x7f82000fd290 msgr2=0x7f82000fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 shutdown_connections 2024-09-12T02:40:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:55.233+0000 7f8206d49640 1 -- 172.21.15.46:0/1992369682 wait complete. 2024-09-12T02:40:55.820 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-12T02:40:55.821 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:40:55.821 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:40:55.839 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:40:55.839 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:40:55.898 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-12T02:40:55.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch host add smithi196 2024-09-12T02:40:56.151 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:40:56.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- 172.21.15.46:0/1864217105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 msgr2=0x7f6878106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 --2- 172.21.15.46:0/1864217105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6868009880 tx=0x7f686802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- 172.21.15.46:0/1864217105 shutdown_connections 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 --2- 172.21.15.46:0/1864217105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- 172.21.15.46:0/1864217105 >> 172.21.15.46:0/1864217105 conn(0x7f68780fc1b0 msgr2=0x7f68780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- 172.21.15.46:0/1864217105 shutdown_connections 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- 172.21.15.46:0/1864217105 wait complete. 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 Processor -- start 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.594+0000 7f687dd39640 1 -- start start 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f687dd39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f687dd39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f687810aa40 con 0x7f6878106560 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f68777fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f68777fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48704/0 (socket says 172.21.15.46:48704) 2024-09-12T02:40:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f68777fe640 1 -- 172.21.15.46:0/3802674661 learned_addr learned my addr 172.21.15.46:0/3802674661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f68777fe640 1 -- 172.21.15.46:0/3802674661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6878197590 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f68777fe640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6868009230 tx=0x7f68680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f686803f070 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6868004d10 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6878197820 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.595+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6868035030 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.596+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687819b920 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.596+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6868042070 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.596+0000 7f6874ff9640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 0x7f686003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.597+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f686806ccb0 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.597+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6878101bd0 con 0x7f6878106560 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.597+0000 7f6876ffd640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 0x7f686003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:40:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.597+0000 7f6876ffd640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 0x7f686003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f686c0098c0 tx=0x7f686c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:40:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.601+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6868039350 con 0x7f6878106560 2024-09-12T02:40:56.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:40:56.782+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f68781069c0 con 0x7f68600390c0 2024-09-12T02:40:57.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:56 smithi046 ceph-mon[30951]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:40:58.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:40:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:40:59.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-12T02:40:59.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:40:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: Deploying cephadm binary to smithi196 2024-09-12T02:40:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:40:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:40:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:00.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:59 smithi046 ceph-mon[30951]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:00.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:40:59 smithi046 ceph-mon[30951]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.706+0000 7f6874ff9640 1 -- 172.21.15.46:0/3802674661 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f68781069c0 con 0x7f68600390c0 2024-09-12T02:41:00.707 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-12T02:41:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 msgr2=0x7f686003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 0x7f686003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f686c0098c0 tx=0x7f686c006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 msgr2=0x7f6878197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6868009230 tx=0x7f68680048f0 comp rx=0 tx=0).stop 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 shutdown_connections 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f68600390c0 0x7f686003b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 --2- 172.21.15.46:0/3802674661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6878106560 0x7f6878197050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 >> 172.21.15.46:0/3802674661 conn(0x7f68780fc1b0 msgr2=0x7f6878069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 shutdown_connections 2024-09-12T02:41:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:00.709+0000 7f687dd39640 1 -- 172.21.15.46:0/3802674661 wait complete. 2024-09-12T02:41:01.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:41:01.596 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:41:01.975 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:01.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:01.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: Added host smithi196 2024-09-12T02:41:01.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:01.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:01.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.059+0000 7f744b957640 1 -- 172.21.15.46:0/3083218145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 msgr2=0x7f7444106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.059+0000 7f744b957640 1 --2- 172.21.15.46:0/3083218145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f7444106920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7440009810 tx=0x7f744002a540 comp rx=0 tx=0).stop 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.059+0000 7f744b957640 1 -- 172.21.15.46:0/3083218145 shutdown_connections 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.059+0000 7f744b957640 1 --2- 172.21.15.46:0/3083218145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f7444106920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.059+0000 7f744b957640 1 -- 172.21.15.46:0/3083218145 >> 172.21.15.46:0/3083218145 conn(0x7f7444077310 msgr2=0x7f74440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 -- 172.21.15.46:0/3083218145 shutdown_connections 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 -- 172.21.15.46:0/3083218145 wait complete. 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 Processor -- start 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 -- start start 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.060+0000 7f744b957640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f744410aa20 con 0x7f7444106540 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f74496f3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f74496f3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48718/0 (socket says 172.21.15.46:48718) 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f74496f3640 1 -- 172.21.15.46:0/986155474 learned_addr learned my addr 172.21.15.46:0/986155474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f74496f3640 1 -- 172.21.15.46:0/986155474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74440ff310 con 0x7f7444106540 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f74496f3640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7440030b00 tx=0x7f7440030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7440040070 con 0x7f7444106540 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74440fd960 con 0x7f7444106540 2024-09-12T02:41:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74400318b0 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7440003de0 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.061+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74440fde20 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.062+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7440039040 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.062+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7444101bb0 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.063+0000 7f743a7fc640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 0x7f743003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.063+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f744006d5e0 con 0x7f7444106540 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.063+0000 7f7448ef2640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 0x7f743003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.063+0000 7f7448ef2640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 0x7f743003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74340098c0 tx=0x7f7434006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.068+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f744003adf0 con 0x7f7444106540 2024-09-12T02:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.248+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74440fe9c0 con 0x7f7430039060 2024-09-12T02:41:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.249+0000 7f743a7fc640 1 -- 172.21.15.46:0/986155474 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f74440fe9c0 con 0x7f7430039060 2024-09-12T02:41:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:41:02.250 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 msgr2=0x7f743003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 0x7f743003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74340098c0 tx=0x7f7434006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 msgr2=0x7f74440fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7440030b00 tx=0x7f7440030b30 comp rx=0 tx=0).stop 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 shutdown_connections 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7430039060 0x7f743003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 --2- 172.21.15.46:0/986155474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7444106540 0x7f74440fd420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 >> 172.21.15.46:0/986155474 conn(0x7f7444077310 msgr2=0x7f74440696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:02.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 shutdown_connections 2024-09-12T02:41:02.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:02.251+0000 7f744b957640 1 -- 172.21.15.46:0/986155474 wait complete. 2024-09-12T02:41:02.887 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T02:41:02.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T02:41:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:41:03.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 -- 172.21.15.46:0/2009739217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 msgr2=0x7fc7b8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 --2- 172.21.15.46:0/2009739217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009880 tx=0x7fc7a802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 -- 172.21.15.46:0/2009739217 shutdown_connections 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 --2- 172.21.15.46:0/2009739217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 -- 172.21.15.46:0/2009739217 >> 172.21.15.46:0/2009739217 conn(0x7fc7b80fd290 msgr2=0x7fc7b80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 -- 172.21.15.46:0/2009739217 shutdown_connections 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.623+0000 7fc7be769640 1 -- 172.21.15.46:0/2009739217 wait complete. 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7be769640 1 Processor -- start 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7be769640 1 -- start start 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7be769640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7be769640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810bb40 con 0x7fc7b8107660 2024-09-12T02:41:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48736/0 (socket says 172.21.15.46:48736) 2024-09-12T02:41:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7b7fff640 1 -- 172.21.15.46:0/2743668607 learned_addr learned my addr 172.21.15.46:0/2743668607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:41:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.624+0000 7fc7b7fff640 1 -- 172.21.15.46:0/2743668607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b8197560 con 0x7fc7b8107660 2024-09-12T02:41:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7b7fff640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009230 tx=0x7fc7a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7a803f070 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81977f0 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7a8004d10 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7a8035030 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.625+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b819b8f0 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc7a8042070 con 0x7fc7b8107660 2024-09-12T02:41:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8102cd0 con 0x7fc7b8107660 2024-09-12T02:41:03.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7b57fa640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 0x7fc7a003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:03.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc7a806cce0 con 0x7fc7b8107660 2024-09-12T02:41:03.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7b77fe640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 0x7fc7a003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:03.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.626+0000 7fc7b77fe640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 0x7fc7a003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0098c0 tx=0x7fc7ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.631+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7a80372d0 con 0x7fc7b8107660 2024-09-12T02:41:03.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:03.810+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc7b8198380 con 0x7fc7b8107660 2024-09-12T02:41:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.044+0000 7fc7b57fa640 1 -- 172.21.15.46:0/2743668607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc7a8042360 con 0x7fc7b8107660 2024-09-12T02:41:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 msgr2=0x7fc7a003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 0x7fc7a003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0098c0 tx=0x7fc7ac006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 msgr2=0x7fc7b8197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009230 tx=0x7fc7a80048f0 comp rx=0 tx=0).stop 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 shutdown_connections 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7a0039060 0x7fc7a003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 --2- 172.21.15.46:0/2743668607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b8107660 0x7fc7b8197020 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 >> 172.21.15.46:0/2743668607 conn(0x7fc7b80fd290 msgr2=0x7fc7b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 shutdown_connections 2024-09-12T02:41:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:04.047+0000 7fc7be769640 1 -- 172.21.15.46:0/2743668607 wait complete. 2024-09-12T02:41:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:04 smithi046 ceph-mon[30951]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:04 smithi046 ceph-mon[30951]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:41:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:04 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:41:04.772 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-09-12T02:41:04.772 INFO:tasks.cephadm:Adding mon.b on smithi145 2024-09-12T02:41:04.772 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-09-12T02:41:04.772 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c' 2024-09-12T02:41:05.016 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:41:05.062 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 -- 172.21.15.196:0/2573417808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 msgr2=0x7fe0d81044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/2573417808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d81044c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009880 tx=0x7fe0d402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 -- 172.21.15.196:0/2573417808 shutdown_connections 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/2573417808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d81044c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 -- 172.21.15.196:0/2573417808 >> 172.21.15.196:0/2573417808 conn(0x7fe0d80fbcd0 msgr2=0x7fe0d80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 -- 172.21.15.196:0/2573417808 shutdown_connections 2024-09-12T02:41:05.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.553+0000 7fe0df6b6640 1 -- 172.21.15.196:0/2573417808 wait complete. 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0df6b6640 1 Processor -- start 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0df6b6640 1 -- start start 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0df6b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0df6b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8107520 con 0x7fe0d81020d0 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0dd452640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0dd452640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.196:55540/0 (socket says 172.21.15.196:55540) 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.554+0000 7fe0dd452640 1 -- 172.21.15.196:0/811322860 learned_addr learned my addr 172.21.15.196:0/811322860 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0dd452640 1 -- 172.21.15.196:0/811322860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d8197170 con 0x7fe0d81020d0 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0dd452640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009230 tx=0x7fe0d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:05.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0d403f070 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d4003aa0 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0d4035030 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d8197460 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.555+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d819a0f0 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.556+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe0d4042070 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.556+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d8107ca0 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.556+0000 7fe0ce7fc640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 0x7fe0c403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.557+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe0d406cef0 con 0x7fe0d81020d0 2024-09-12T02:41:05.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.557+0000 7fe0dcc51640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 0x7fe0c403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:05.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.557+0000 7fe0dcc51640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 0x7fe0c403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80098c0 tx=0x7fe0c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:05.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.562+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0d40433b0 con 0x7fe0d81020d0 2024-09-12T02:41:05.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.750+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fe0d802d9a0 con 0x7fe0c4039060 2024-09-12T02:41:05.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.756+0000 7fe0ce7fc640 1 -- 172.21.15.196:0/811322860 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe0d802d9a0 con 0x7fe0c4039060 2024-09-12T02:41:05.755 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 msgr2=0x7fe0c403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 0x7fe0c403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80098c0 tx=0x7fe0c8006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 msgr2=0x7fe0d8196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009230 tx=0x7fe0d40048f0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 shutdown_connections 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c4039060 0x7fe0c403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 --2- 172.21.15.196:0/811322860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d81020d0 0x7fe0d8196c30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:05.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 >> 172.21.15.196:0/811322860 conn(0x7fe0d80fbcd0 msgr2=0x7fe0d80fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:05.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 shutdown_connections 2024-09-12T02:41:05.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:05.758+0000 7fe0df6b6640 1 -- 172.21.15.196:0/811322860 wait complete. 2024-09-12T02:41:06.326 DEBUG:teuthology.orchestra.run.smithi145:mon.b> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T02:41:06.329 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T02:41:06.331 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:41:06.331 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:41:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:06.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:06 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:07.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:07 smithi046 ceph-mon[30951]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:07.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:07 smithi046 ceph-mon[30951]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:07.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:07 smithi046 ceph-mon[30951]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:07.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:07 smithi046 ceph-mon[30951]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c;count:3 2024-09-12T02:41:07.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:07 smithi046 ceph-mon[30951]: Deploying daemon mon.c on smithi196 2024-09-12T02:41:07.484 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:41:08.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:08 smithi046 ceph-mon[30951]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 -- 172.21.15.196:0/3048091466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 msgr2=0x7f6240106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 --2- 172.21.15.196:0/3048091466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6234009810 tx=0x7f623402a540 comp rx=0 tx=0).stop 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 -- 172.21.15.196:0/3048091466 shutdown_connections 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 --2- 172.21.15.196:0/3048091466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 -- 172.21.15.196:0/3048091466 >> 172.21.15.196:0/3048091466 conn(0x7f6240077310 msgr2=0x7f62400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 -- 172.21.15.196:0/3048091466 shutdown_connections 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.814+0000 7f6246f64640 1 -- 172.21.15.196:0/3048091466 wait complete. 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.815+0000 7f6246f64640 1 Processor -- start 2024-09-12T02:41:08.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.815+0000 7f6246f64640 1 -- start start 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.815+0000 7f6246f64640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.815+0000 7f6246f64640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f624010aa20 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6244d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6244d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.196:55556/0 (socket says 172.21.15.196:55556) 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6244d00640 1 -- 172.21.15.196:0/4203965354 learned_addr learned my addr 172.21.15.196:0/4203965354 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6244d00640 1 -- 172.21.15.196:0/4203965354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401983d0 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6244d00640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6234030b00 tx=0x7f6234030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6234040070 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6240198660 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62340318b0 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.816+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6234003de0 con 0x7f6240106540 2024-09-12T02:41:08.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.817+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6240198ac0 con 0x7f6240106540 2024-09-12T02:41:08.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.817+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6234039040 con 0x7f6240106540 2024-09-12T02:41:08.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.818+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6240101bb0 con 0x7f6240106540 2024-09-12T02:41:08.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.818+0000 7f623dffb640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 0x7f623003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:08.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.818+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f623406e490 con 0x7f6240106540 2024-09-12T02:41:08.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.818+0000 7f623ffff640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 0x7f623003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:08.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.819+0000 7f623ffff640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 0x7f623003b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098c0 tx=0x7f622c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:08.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:08.823+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6234043bd0 con 0x7f6240106540 2024-09-12T02:41:09.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.073+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6240106920 con 0x7f6240106540 2024-09-12T02:41:09.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.074+0000 7f623dffb640 1 -- 172.21.15.196:0/4203965354 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f623404d3a0 con 0x7f6240106540 2024-09-12T02:41:09.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:41:09.078 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","modified":"2024-09-12T02:39:43.672013Z","created":"2024-09-12T02:39:43.672013Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:41:09.078 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-12T02:41:09.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 msgr2=0x7f623003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:09.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 0x7f623003b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098c0 tx=0x7f622c006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:09.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 msgr2=0x7f6240197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:09.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6234030b00 tx=0x7f6234030b30 comp rx=0 tx=0).stop 2024-09-12T02:41:09.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 shutdown_connections 2024-09-12T02:41:09.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62300390c0 0x7f623003b580 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098c0 tx=0x7f622c006db0 comp rx=0 tx=0).stop 2024-09-12T02:41:09.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 --2- 172.21.15.196:0/4203965354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6240106540 0x7f6240197e90 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6234030b00 tx=0x7f6234030b30 comp rx=0 tx=0).stop 2024-09-12T02:41:09.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.085+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 >> 172.21.15.196:0/4203965354 conn(0x7f6240077310 msgr2=0x7f62400feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:09.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.086+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 shutdown_connections 2024-09-12T02:41:09.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:09.086+0000 7f6246f64640 1 -- 172.21.15.196:0/4203965354 wait complete. 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: btime 2024-09-12T02:39:49:303263+0000 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-12T02:41:10.324 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-09-12T02:41:10.325 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:41:10.325 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:41:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:10 smithi046 ceph-mon[30951]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:10 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/4203965354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:10.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Added host smithi196 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:41:10.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c;count:3 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: Deploying daemon mon.c on smithi196 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/4203965354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 ceph-mon[34587]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:41:10.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 bash[34568]: 58fda31218fb03bfc9e31a9ce88943513b7b248dc828d2ee0f2ebc3b7aceb29f 2024-09-12T02:41:10.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:10 smithi196 systemd[1]: Started Ceph mon.c for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:41:10.898 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:41:10.899 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:41:11.146 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:41:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: mon.a calling monitor election 2024-09-12T02:41:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: mon.c calling monitor election 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: monmap epoch 2 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: last_changed 2024-09-12T02:41:10.329090+0000 2024-09-12T02:41:15.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: min_mon_release 19 (squid) 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: election_strategy: 1 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: fsmap 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: mgrmap e13: x(active, since 39s) 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: overall HEALTH_OK 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: mon.a calling monitor election 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: mon.c calling monitor election 2024-09-12T02:41:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: monmap epoch 2 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: last_changed 2024-09-12T02:41:10.329090+0000 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: min_mon_release 19 (squid) 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: election_strategy: 1 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: fsmap 2024-09-12T02:41:15.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: mgrmap e13: x(active, since 39s) 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: overall HEALTH_OK 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:15.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.968+0000 7fabdb38f640 1 -- 172.21.15.196:0/1418282919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabc0005740 msgr2=0x7fabc0007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.968+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1418282919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabc0005740 0x7fabc0007bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd41016c0 tx=0x7fabd002c490 comp rx=0 tx=0).stop 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.968+0000 7fabdb38f640 1 -- 172.21.15.196:0/1418282919 shutdown_connections 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.968+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1418282919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabc0005740 0x7fabc0007bd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.968+0000 7fabdb38f640 1 -- 172.21.15.196:0/1418282919 >> 172.21.15.196:0/1418282919 conn(0x7fabd40fc280 msgr2=0x7fabd40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.969+0000 7fabdb38f640 1 -- 172.21.15.196:0/1418282919 shutdown_connections 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.969+0000 7fabdb38f640 1 -- 172.21.15.196:0/1418282919 wait complete. 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.969+0000 7fabdb38f640 1 Processor -- start 2024-09-12T02:41:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabdb38f640 1 -- start start 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabdb38f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabdb38f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 0x7fabd419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabdb38f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4107570 con 0x7fabd4199920 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabdb38f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41073f0 con 0x7fabc0005740 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56008/0 (socket says 172.21.15.196:56008) 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 -- 172.21.15.196:0/1472035531 learned_addr learned my addr 172.21.15.196:0/1472035531 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 -- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 msgr2=0x7fabd41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd892a640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 0x7fabd419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 -- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 msgr2=0x7fabd41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:41:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.970+0000 7fabd912b640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabd892a640 1 -- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 msgr2=0x7fabd41993e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabd892a640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabd892a640 1 -- 172.21.15.196:0/1472035531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd4199f40 con 0x7fabd4199920 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabd892a640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 0x7fabd419dcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b3d0 tx=0x7fabc400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fabc4011630 con 0x7fabd4199920 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc4011c80 con 0x7fabd4199920 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd419e1f0 con 0x7fabd4199920 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fabc4010d80 con 0x7fabd4199920 2024-09-12T02:41:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.971+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd419e7b0 con 0x7fabd4199920 2024-09-12T02:41:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.972+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fabc4018020 con 0x7fabd4199920 2024-09-12T02:41:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.973+0000 7fabca7fc640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 0x7fabb803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.973+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd419eac0 con 0x7fabd4199920 2024-09-12T02:41:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.973+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fabc404bc20 con 0x7fabd4199920 2024-09-12T02:41:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.973+0000 7fabd912b640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 0x7fabb803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.973+0000 7fabd912b640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 0x7fabb803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabd0012c10 tx=0x7fabd00323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:15.978+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabc404be20 con 0x7fabd4199920 2024-09-12T02:41:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.215+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fabd419a770 con 0x7fabd4199920 2024-09-12T02:41:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.216+0000 7fabca7fc640 1 -- 172.21.15.196:0/1472035531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fabc401f020 con 0x7fabd4199920 2024-09-12T02:41:16.217 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:41:16.217 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","modified":"2024-09-12T02:41:10.329090Z","created":"2024-09-12T02:39:43.672013Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:41:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 msgr2=0x7fabb803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 0x7fabb803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabd0012c10 tx=0x7fabd00323a0 comp rx=0 tx=0).stop 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 msgr2=0x7fabd419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 0x7fabd419dcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b3d0 tx=0x7fabc400b890 comp rx=0 tx=0).stop 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 shutdown_connections 2024-09-12T02:41:16.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabd4199920 0x7fabd419dcb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fabb8038e50 0x7fabb803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 --2- 172.21.15.196:0/1472035531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabc0005740 0x7fabd41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 >> 172.21.15.196:0/1472035531 conn(0x7fabd40fc280 msgr2=0x7fabd41049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 shutdown_connections 2024-09-12T02:41:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:16.218+0000 7fabdb38f640 1 -- 172.21.15.196:0/1472035531 wait complete. 2024-09-12T02:41:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:16 smithi196 ceph-mon[34587]: Deploying daemon mon.b on smithi145 2024-09-12T02:41:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:16 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/1472035531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:16 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:16.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:16 smithi046 ceph-mon[30951]: Deploying daemon mon.b on smithi145 2024-09-12T02:41:16.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:16 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/1472035531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:16.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:16 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:17 smithi196 ceph-mon[34587]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:17.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:17 smithi046 ceph-mon[30951]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:18.038 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:41:18.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:41:18.286 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:41:18.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 -- 172.21.15.196:0/1877106766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 msgr2=0x7fc80c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 --2- 172.21.15.196:0/1877106766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009880 tx=0x7fc7fc02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 -- 172.21.15.196:0/1877106766 shutdown_connections 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 --2- 172.21.15.196:0/1877106766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 -- 172.21.15.196:0/1877106766 >> 172.21.15.196:0/1877106766 conn(0x7fc80c0fd320 msgr2=0x7fc80c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 -- 172.21.15.196:0/1877106766 shutdown_connections 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.668+0000 7fc81192c640 1 -- 172.21.15.196:0/1877106766 wait complete. 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 Processor -- start 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 -- start start 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c10bbd0 con 0x7fc80c107660 2024-09-12T02:41:18.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc81192c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c10ba50 con 0x7fc80c074a30 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56018/0 (socket says 172.21.15.196:56018) 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 -- 172.21.15.196:0/2591360367 learned_addr learned my addr 172.21.15.196:0/2591360367 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80affd640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 -- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 msgr2=0x7fc80c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 -- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 msgr2=0x7fc80c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.669+0000 7fc80a7fc640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc80affd640 1 -- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 msgr2=0x7fc80c074e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc80affd640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc80affd640 1 -- 172.21.15.196:0/2591360367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c0763e0 con 0x7fc80c107660 2024-09-12T02:41:18.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc80affd640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c0744f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc006010 tx=0x7fc7fc004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc7fc035030 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7fc0053a0 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1a34d0 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc7fc031830 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.670+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a3a60 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.671+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c102da0 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.671+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc7fc042070 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.671+0000 7fc81092a640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 0x7fc7f403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.671+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc7fc06cd10 con 0x7fc80c107660 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.672+0000 7fc80a7fc640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 0x7fc7f403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.672+0000 7fc80a7fc640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 0x7fc7f403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc800009100 tx=0x7fc800008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.676+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7fc0372d0 con 0x7fc80c107660 2024-09-12T02:41:18.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.902+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc80c1a3d10 con 0x7fc80c107660 2024-09-12T02:41:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.903+0000 7fc81092a640 1 -- 172.21.15.196:0/2591360367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc7fc005550 con 0x7fc80c107660 2024-09-12T02:41:18.904 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:41:18.904 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","modified":"2024-09-12T02:41:10.329090Z","created":"2024-09-12T02:39:43.672013Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:41:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 msgr2=0x7fc7f403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 0x7fc7f403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc800009100 tx=0x7fc800008040 comp rx=0 tx=0).stop 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 msgr2=0x7fc80c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c0744f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc006010 tx=0x7fc7fc004e40 comp rx=0 tx=0).stop 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 shutdown_connections 2024-09-12T02:41:18.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc7f4038e00 0x7fc7f403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80c074a30 0x7fc80c074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 --2- 172.21.15.196:0/2591360367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c107660 0x7fc80c0744f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 >> 172.21.15.196:0/2591360367 conn(0x7fc80c0fd320 msgr2=0x7fc80c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 shutdown_connections 2024-09-12T02:41:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:18.905+0000 7fc81192c640 1 -- 172.21.15.196:0/2591360367 wait complete. 2024-09-12T02:41:19.370 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(???) e0 preinit fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:19.370 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T02:41:19.370 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T02:41:19.370 INFO:journalctl@ceph.mon.b.smithi145.stdout: e1 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: btime 2024-09-12T02:39:49:303263+0000 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: legacy client fscid: -1 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout: No filesystems configured 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:41:19.371 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Added host smithi196 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.372 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2743668607' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:41:19.373 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c;count:3 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Deploying daemon mon.c on smithi196 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.374 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/4203965354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.a calling monitor election 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.c calling monitor election 2024-09-12T02:41:19.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: monmap epoch 2 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: last_changed 2024-09-12T02:41:10.329090+0000 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: min_mon_release 19 (squid) 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: election_strategy: 1 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:19.376 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: fsmap 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mgrmap e13: x(active, since 39s) 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: overall HEALTH_OK 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: Deploying daemon mon.b on smithi145 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/1472035531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:19.377 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 ceph-mon[34674]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:41:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 bash[34655]: be1aba72babc1a1d3b1a7ac9bc34b27117b60efade7af84fe46f309978d74500 2024-09-12T02:41:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:19 smithi145 systemd[1]: Started Ceph mon.b for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:41:20.481 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:41:20.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:41:20.732 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: mon.c calling monitor election 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: mon.a calling monitor election 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: monmap epoch 3 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:24.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: min_mon_release 19 (squid) 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: election_strategy: 1 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: fsmap 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: mgrmap e13: x(active, since 48s) 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: overall HEALTH_OK 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 -- 172.21.15.196:0/1719564859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce4003530 msgr2=0x7f7ce40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:24.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 --2- 172.21.15.196:0/1719564859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce4003530 0x7f7ce40059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc102af0 tx=0x7f7cf402c3f0 comp rx=0 tx=0).stop 2024-09-12T02:41:24.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 -- 172.21.15.196:0/1719564859 shutdown_connections 2024-09-12T02:41:24.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 --2- 172.21.15.196:0/1719564859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce4003530 0x7f7ce40059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:24.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 -- 172.21.15.196:0/1719564859 >> 172.21.15.196:0/1719564859 conn(0x7f7cfc0fd320 msgr2=0x7f7cfc0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:24.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 -- 172.21.15.196:0/1719564859 shutdown_connections 2024-09-12T02:41:24.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.768+0000 7f7d032e0640 1 -- 172.21.15.196:0/1719564859 wait complete. 2024-09-12T02:41:24.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.769+0000 7f7d032e0640 1 Processor -- start 2024-09-12T02:41:24.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.769+0000 7f7d032e0640 1 -- start start 2024-09-12T02:41:24.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.769+0000 7f7d032e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:24.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.769+0000 7f7d032e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:24.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.769+0000 7f7d032e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 0x7f7cfc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d032e0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc109630 con 0x7f7cfc19c110 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d032e0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc1094b0 con 0x7f7ce4003530 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d032e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc1097b0 con 0x7f7cfc197840 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56020/0 (socket says 172.21.15.196:56020) 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 -- 172.21.15.196:0/3836092882 learned_addr learned my addr 172.21.15.196:0/3836092882 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0107c640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 0x7f7cfc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 msgr2=0x7f7cfc19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 msgr2=0x7f7cfc19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:41:24.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0087b640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0107c640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 msgr2=0x7f7cfc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0107c640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 msgr2=0x7f7cfc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0107c640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0107c640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 msgr2=0x7f7cfc19bbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 msgr2=0x7f7cfc197300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 -- 172.21.15.196:0/3836092882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc1a3e80 con 0x7f7cfc19c110 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d0187d640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 0x7f7cfc19c5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009fd0 tx=0x7f7cec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec015020 con 0x7f7cfc19c110 2024-09-12T02:41:24.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.770+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc1a40b0 con 0x7f7cfc19c110 2024-09-12T02:41:24.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.771+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cec011ce0 con 0x7f7cfc19c110 2024-09-12T02:41:24.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.771+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec010e40 con 0x7f7cfc19c110 2024-09-12T02:41:24.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.771+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc1a45c0 con 0x7f7cfc19c110 2024-09-12T02:41:24.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.771+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc10a6e0 con 0x7f7cfc19c110 2024-09-12T02:41:24.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.772+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7cec018020 con 0x7f7cfc19c110 2024-09-12T02:41:24.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.772+0000 7f7cfa7fc640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 0x7f7ce803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:24.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.772+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7cec04bea0 con 0x7f7cfc19c110 2024-09-12T02:41:24.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.772+0000 7f7d0107c640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 0x7f7ce803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:24.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.773+0000 7f7d0107c640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 0x7f7ce803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4014040 tx=0x7f7cf4014930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:24.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:24.777+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cec01f020 con 0x7f7cfc19c110 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: mon.c calling monitor election 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: mon.a calling monitor election 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: monmap epoch 3 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: min_mon_release 19 (squid) 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: election_strategy: 1 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: fsmap 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: mgrmap e13: x(active, since 48s) 2024-09-12T02:41:24.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: overall HEALTH_OK 2024-09-12T02:41:24.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:24.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:25.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.025+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7cfc190780 con 0x7f7cfc19c110 2024-09-12T02:41:25.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.026+0000 7f7cfa7fc640 1 -- 172.21.15.196:0/3836092882 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f7cec04d020 con 0x7f7cfc19c110 2024-09-12T02:41:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:41:25.027 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","modified":"2024-09-12T02:41:19.374976Z","created":"2024-09-12T02:39:43.672013Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:41:25.027 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-12T02:41:25.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 msgr2=0x7f7ce803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 0x7f7ce803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4014040 tx=0x7f7cf4014930 comp rx=0 tx=0).stop 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 msgr2=0x7f7cfc19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 0x7f7cfc19c5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009fd0 tx=0x7f7cec00b870 comp rx=0 tx=0).stop 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 shutdown_connections 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc19c110 0x7f7cfc19c5c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:25.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7ce80394f0 0x7f7ce803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:25.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cfc197840 0x7f7cfc19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:25.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 --2- 172.21.15.196:0/3836092882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ce4003530 0x7f7cfc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:25.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 >> 172.21.15.196:0/3836092882 conn(0x7f7cfc0fd320 msgr2=0x7f7cfc106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:25.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 shutdown_connections 2024-09-12T02:41:25.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:25.028+0000 7f7d032e0640 1 -- 172.21.15.196:0/3836092882 wait complete. 2024-09-12T02:41:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:25 smithi196 ceph-mon[34587]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:25 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/3836092882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:25.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:25 smithi046 ceph-mon[30951]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:25 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/3836092882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.307 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T02:41:26.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T02:41:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:41:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:41:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: mon.c calling monitor election 2024-09-12T02:41:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: mon.a calling monitor election 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: monmap epoch 3 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:26.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: min_mon_release 19 (squid) 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: election_strategy: 1 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: fsmap 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: mgrmap e13: x(active, since 48s) 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: overall HEALTH_OK 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:26.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:26.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:26.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/3836092882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:41:26.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:27.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 -- 172.21.15.46:0/1520586061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b181027b0 msgr2=0x7f3b18102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:27.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 --2- 172.21.15.46:0/1520586061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b181027b0 0x7f3b18102b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009880 tx=0x7f3b1402a670 comp rx=0 tx=0).stop 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 -- 172.21.15.46:0/1520586061 shutdown_connections 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 --2- 172.21.15.46:0/1520586061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b181027b0 0x7f3b18102b90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 -- 172.21.15.46:0/1520586061 >> 172.21.15.46:0/1520586061 conn(0x7f3b180fc1b0 msgr2=0x7f3b180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.154+0000 7f3b205a6640 1 -- 172.21.15.46:0/1520586061 shutdown_connections 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.155+0000 7f3b205a6640 1 -- 172.21.15.46:0/1520586061 wait complete. 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.155+0000 7f3b205a6640 1 Processor -- start 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.155+0000 7f3b205a6640 1 -- start start 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 0x7f3b18197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b181978c0 0x7f3b1819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810bb10 con 0x7f3b1819c190 2024-09-12T02:41:27.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810b990 con 0x7f3b181027b0 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1810bc90 con 0x7f3b181978c0 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45354/0 (socket says 172.21.15.46:45354) 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 -- 172.21.15.46:0/4082125370 learned_addr learned my addr 172.21.15.46:0/4082125370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1e342640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 0x7f3b18197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1db41640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b181978c0 0x7f3b1819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 -- 172.21.15.46:0/4082125370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b181978c0 msgr2=0x7f3b1819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b181978c0 0x7f3b1819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 -- 172.21.15.46:0/4082125370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 msgr2=0x7f3b18197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 0x7f3b18197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 -- 172.21.15.46:0/4082125370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181a3f00 con 0x7f3b1819c190 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1e342640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 0x7f3b18197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:41:27.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b1eb43640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000ba50 tx=0x7f3b1000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10019070 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b1000cbe0 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b181a4130 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.156+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b100075d0 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.157+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181a4640 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.157+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3b1001c070 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.157+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1806a750 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.158+0000 7f3b0f7fe640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 0x7f3b0403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.158+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3b1004bc90 con 0x7f3b1819c190 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.158+0000 7f3b1e342640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 0x7f3b0403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.158+0000 7f3b1e342640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 0x7f3b0403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005ab0 tx=0x7f3b14005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:27.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.162+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b100112a0 con 0x7f3b1819c190 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.344+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3b18107e10 con 0x7f3b1819c190 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.344+0000 7f3b0f7fe640 1 -- 172.21.15.46:0/4082125370 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f3b1000ce00 con 0x7f3b1819c190 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:27.346 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-12T02:41:27.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.347+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 msgr2=0x7f3b0403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:27.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.347+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 0x7f3b0403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005ab0 tx=0x7f3b14005a20 comp rx=0 tx=0).stop 2024-09-12T02:41:27.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.347+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 msgr2=0x7f3b1819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:27.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.347+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000ba50 tx=0x7f3b1000bf10 comp rx=0 tx=0).stop 2024-09-12T02:41:27.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 shutdown_connections 2024-09-12T02:41:27.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1819c190 0x7f3b1819c640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3b040394f0 0x7f3b0403b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b181978c0 0x7f3b1819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 --2- 172.21.15.46:0/4082125370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181027b0 0x7f3b18197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 >> 172.21.15.46:0/4082125370 conn(0x7f3b180fc1b0 msgr2=0x7f3b18105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 shutdown_connections 2024-09-12T02:41:27.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:27.348+0000 7f3b205a6640 1 -- 172.21.15.46:0/4082125370 wait complete. 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mon.b calling monitor election 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: Reconfiguring daemon mon.a on smithi046 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mon.b calling monitor election 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mon.c calling monitor election 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mon.a calling monitor election 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:41:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: monmap epoch 3 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: min_mon_release 19 (squid) 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: election_strategy: 1 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: fsmap 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: mgrmap e13: x(active, since 50s) 2024-09-12T02:41:27.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:27 smithi046 ceph-mon[30951]: overall HEALTH_OK 2024-09-12T02:41:27.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mon.b calling monitor election 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: Reconfiguring daemon mon.a on smithi046 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mon.b calling monitor election 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mon.c calling monitor election 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mon.a calling monitor election 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:41:27.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: monmap epoch 3 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: min_mon_release 19 (squid) 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: election_strategy: 1 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:27.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:27.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: fsmap 2024-09-12T02:41:27.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:27.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: mgrmap e13: x(active, since 50s) 2024-09-12T02:41:27.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:27 smithi145 ceph-mon[34674]: overall HEALTH_OK 2024-09-12T02:41:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mon.b calling monitor election 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: Reconfiguring daemon mon.a on smithi046 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mon.b calling monitor election 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mon.c calling monitor election 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mon.a calling monitor election 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: monmap epoch 3 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: last_changed 2024-09-12T02:41:19.374976+0000 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: created 2024-09-12T02:39:43.672013+0000 2024-09-12T02:41:27.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: min_mon_release 19 (squid) 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: election_strategy: 1 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: 2: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: fsmap 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: mgrmap e13: x(active, since 50s) 2024-09-12T02:41:27.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:27 smithi196 ceph-mon[34587]: overall HEALTH_OK 2024-09-12T02:41:27.890 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T02:41:27.890 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:41:27.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:41:27.919 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:41:27.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:27.986 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:41:27.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:41:28.019 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:41:28.019 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:28.088 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:41:28.088 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:41:28.120 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:41:28.120 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:41:28.192 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-09-12T02:41:28.193 INFO:tasks.cephadm:Adding mgr.y on smithi145 2024-09-12T02:41:28.193 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-09-12T02:41:28.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi145=y;smithi196=z' 2024-09-12T02:41:28.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:28 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4082125370' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:28.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:28.479 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:41:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:28 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4082125370' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:28.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:41:28 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:41:28.374+0000 7f0ab7099640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T02:41:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:28 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4082125370' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:41:28.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 -- 172.21.15.196:0/2664620266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc1016e0 msgr2=0x7f5dcc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:28.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 --2- 172.21.15.196:0/2664620266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc101ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009880 tx=0x7f5dbc02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:41:28.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 -- 172.21.15.196:0/2664620266 shutdown_connections 2024-09-12T02:41:28.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 --2- 172.21.15.196:0/2664620266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc101ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 -- 172.21.15.196:0/2664620266 >> 172.21.15.196:0/2664620266 conn(0x7f5dcc0fce90 msgr2=0x7f5dcc0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 -- 172.21.15.196:0/2664620266 shutdown_connections 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.913+0000 7f5dd2277640 1 -- 172.21.15.196:0/2664620266 wait complete. 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 Processor -- start 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 -- start start 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 0x7f5dcc074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 0x7f5dcc1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc107680 con 0x7f5dcc0745e0 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc107500 con 0x7f5dcc074fa0 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dd2277640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc107800 con 0x7f5dcc1016e0 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:28.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52270/0 (socket says 172.21.15.196:52270) 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dcbfff640 1 -- 172.21.15.196:0/133770884 learned_addr learned my addr 172.21.15.196:0/133770884 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.914+0000 7f5dc3fff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 0x7f5dcc074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dd0814640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 0x7f5dcc1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 -- 172.21.15.196:0/133770884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 msgr2=0x7f5dcc1a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 0x7f5dcc1a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 -- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 msgr2=0x7f5dcc074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 0x7f5dcc074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 -- 172.21.15.196:0/133770884 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc1a1500 con 0x7f5dcc1016e0 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dc3fff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 0x7f5dcc074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dd0814640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 0x7f5dcc1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dcbfff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc006010 tx=0x7f5dbc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc03f030 con 0x7f5dcc1016e0 2024-09-12T02:41:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc1a1790 con 0x7f5dcc1016e0 2024-09-12T02:41:28.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dbc003cb0 con 0x7f5dcc1016e0 2024-09-12T02:41:28.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc035070 con 0x7f5dcc1016e0 2024-09-12T02:41:28.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.915+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a1d40 con 0x7f5dcc1016e0 2024-09-12T02:41:28.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.916+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5dbc042030 con 0x7f5dcc1016e0 2024-09-12T02:41:28.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.916+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc190440 con 0x7f5dcc1016e0 2024-09-12T02:41:28.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.916+0000 7f5dc9ffb640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 0x7f5db803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:28.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.916+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5dbc06ce20 con 0x7f5dcc1016e0 2024-09-12T02:41:28.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.916+0000 7f5dc3fff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 0x7f5db803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:28.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.917+0000 7f5dc3fff640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 0x7f5db803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc075d10 tx=0x7f5db4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:28.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:28.921+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dbc0303a0 con 0x7f5dcc1016e0 2024-09-12T02:41:29.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.101+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi145=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f5dcc062630 con 0x7f5db80394f0 2024-09-12T02:41:29.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.106+0000 7f5dc9ffb640 1 -- 172.21.15.196:0/133770884 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5dcc062630 con 0x7f5db80394f0 2024-09-12T02:41:29.106 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 msgr2=0x7f5db803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 0x7f5db803b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc075d10 tx=0x7f5db4009190 comp rx=0 tx=0).stop 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 msgr2=0x7f5dcc075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc006010 tx=0x7f5dbc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 shutdown_connections 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dcc074fa0 0x7f5dcc1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dcc0745e0 0x7f5dcc074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:29.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f5db80394f0 0x7f5db803b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:29.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 --2- 172.21.15.196:0/133770884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc1016e0 0x7f5dcc075f90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:29.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 >> 172.21.15.196:0/133770884 conn(0x7f5dcc0fce90 msgr2=0x7f5dcc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:29.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 shutdown_connections 2024-09-12T02:41:29.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:41:29.108+0000 7f5dd2277640 1 -- 172.21.15.196:0/133770884 wait complete. 2024-09-12T02:41:29.758 DEBUG:teuthology.orchestra.run.smithi145:mgr.y> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T02:41:29.760 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T02:41:29.763 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T02:41:29.763 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:41:29.763 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:41:29.783 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:41:29.783 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-12T02:41:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:41:29.842 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:29.842 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T02:41:29.842 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:29.842 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:29.843 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 02:40:46.652180824 +0000 2024-09-12T02:41:29.843 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 02:36:56.588477425 +0000 2024-09-12T02:41:29.843 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 02:36:56.588477425 +0000 2024-09-12T02:41:29.843 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 02:36:56.588477425 +0000 2024-09-12T02:41:29.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:41:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T02:41:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T02:41:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000132359 s, 3.9 MB/s 2024-09-12T02:41:29.913 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:41:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: Reconfiguring daemon mon.b on smithi145 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi145=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: Saving service mgr spec with placement smithi046=x;smithi145=y;smithi196=z;count:3 2024-09-12T02:41:29.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:29.973 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-12T02:41:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:41:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 02:40:46.688180435 +0000 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 02:36:56.922473907 +0000 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 02:36:56.922473907 +0000 2024-09-12T02:41:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 02:36:56.922473907 +0000 2024-09-12T02:41:30.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:30.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: Reconfiguring daemon mon.b on smithi145 2024-09-12T02:41:30.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi145=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:30.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: Saving service mgr spec with placement smithi046=x;smithi145=y;smithi196=z;count:3 2024-09-12T02:41:30.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T02:41:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T02:41:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000112389 s, 4.6 MB/s 2024-09-12T02:41:30.099 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:41:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: Reconfiguring daemon mon.b on smithi145 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi145=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:30.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: Saving service mgr spec with placement smithi046=x;smithi145=y;smithi196=z;count:3 2024-09-12T02:41:30.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:30.157 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-12T02:41:30.214 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 02:40:46.726180024 +0000 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 02:36:57.248470474 +0000 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 02:36:57.248470474 +0000 2024-09-12T02:41:30.215 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 02:36:57.248470474 +0000 2024-09-12T02:41:30.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:41:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T02:41:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T02:41:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000119085 s, 4.3 MB/s 2024-09-12T02:41:30.284 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:41:30.342 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-12T02:41:30.399 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:41:30.399 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 02:40:46.761179646 +0000 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 02:36:57.561467178 +0000 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 02:36:57.561467178 +0000 2024-09-12T02:41:30.400 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-12 02:36:57.561467178 +0000 2024-09-12T02:41:30.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:41:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-12T02:41:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-12T02:41:30.464 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000122154 s, 4.2 MB/s 2024-09-12T02:41:30.465 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:41:30.522 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-12T02:41:30.522 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:41:30.540 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:41:30.540 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-09-12T02:41:30.595 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-12 02:41:18.839095426 +0000 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-12 02:36:53.073369100 +0000 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-12 02:36:53.073369100 +0000 2024-09-12T02:41:30.596 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-12 02:36:53.073369100 +0000 2024-09-12T02:41:30.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:41:30.660 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-12T02:41:30.660 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-12T02:41:30.660 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000235145 s, 2.2 MB/s 2024-09-12T02:41:30.661 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:41:30.718 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-09-12T02:41:30.773 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:41:30.773 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-12 02:41:18.839095426 +0000 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-12 02:36:53.416361660 +0000 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-12 02:36:53.416361660 +0000 2024-09-12T02:41:30.774 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-12 02:36:53.416361660 +0000 2024-09-12T02:41:30.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:41:30.836 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-12T02:41:30.837 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-12T02:41:30.837 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000225816 s, 2.3 MB/s 2024-09-12T02:41:30.838 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:41:30.894 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:30.949 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-12 02:41:18.840095404 +0000 2024-09-12T02:41:30.950 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-12 02:36:53.733354784 +0000 2024-09-12T02:41:30.950 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-12 02:36:53.733354784 +0000 2024-09-12T02:41:30.950 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-12 02:36:53.733354784 +0000 2024-09-12T02:41:30.950 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:41:31.012 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-12T02:41:31.012 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-12T02:41:31.012 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000192527 s, 2.7 MB/s 2024-09-12T02:41:31.013 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:41:31.069 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-12 02:41:18.840095404 +0000 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-12 02:36:54.053347843 +0000 2024-09-12T02:41:31.124 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-12 02:36:54.053347843 +0000 2024-09-12T02:41:31.125 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-12 02:36:54.053347843 +0000 2024-09-12T02:41:31.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:41:31.187 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-12T02:41:31.187 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-12T02:41:31.187 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000211466 s, 2.4 MB/s 2024-09-12T02:41:31.188 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:41:31.245 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-12T02:41:31.245 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:41:31.262 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:41:31.262 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-12 02:41:09.653960469 +0000 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-12 02:36:53.076457342 +0000 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-12 02:36:53.076457342 +0000 2024-09-12T02:41:31.319 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-12 02:36:53.076457342 +0000 2024-09-12T02:41:31.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:41:31.383 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-12T02:41:31.383 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-12T02:41:31.383 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000186135 s, 2.8 MB/s 2024-09-12T02:41:31.385 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:41:31.441 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-12T02:41:31.495 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:41:31.495 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:31.495 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-12 02:41:09.653960469 +0000 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-12 02:36:53.410449928 +0000 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-12 02:36:53.410449928 +0000 2024-09-12T02:41:31.496 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-12 02:36:53.410449928 +0000 2024-09-12T02:41:31.496 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:41:31.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:31.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:41:31.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:31.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:31.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:31.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:31 smithi196 ceph-mon[34587]: Reconfiguring daemon mon.c on smithi196 2024-09-12T02:41:31.558 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-12T02:41:31.559 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-12T02:41:31.559 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000178779 s, 2.9 MB/s 2024-09-12T02:41:31.560 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:31 smithi145 ceph-mon[34674]: Reconfiguring daemon mon.c on smithi196 2024-09-12T02:41:31.616 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-12 02:41:09.653960469 +0000 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-12 02:36:53.735442715 +0000 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-12 02:36:53.735442715 +0000 2024-09-12T02:41:31.671 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-12 02:36:53.735442715 +0000 2024-09-12T02:41:31.672 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:41:31.736 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-12T02:41:31.736 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-12T02:41:31.736 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00020143 s, 2.5 MB/s 2024-09-12T02:41:31.737 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:41:31.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:31.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:31 smithi046 ceph-mon[30951]: Reconfiguring daemon mon.c on smithi196 2024-09-12T02:41:31.794 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-12 02:41:09.654960447 +0000 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-12 02:36:54.050435723 +0000 2024-09-12T02:41:31.851 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-12 02:36:54.050435723 +0000 2024-09-12T02:41:31.852 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-12 02:36:54.050435723 +0000 2024-09-12T02:41:31.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:41:31.916 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-12T02:41:31.916 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-12T02:41:31.916 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000235903 s, 2.2 MB/s 2024-09-12T02:41:31.917 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:41:31.975 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-09-12T02:41:31.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:41:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:33.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:32 smithi046 ceph-mon[30951]: Deploying daemon mgr.z on smithi196 2024-09-12T02:41:33.089 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:32 smithi196 ceph-mon[34587]: Deploying daemon mgr.z on smithi196 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:33.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:33.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:33.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:33.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:33.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:32 smithi145 ceph-mon[34674]: Deploying daemon mgr.z on smithi196 2024-09-12T02:41:33.721 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:33 smithi196 systemd[1]: Starting Ceph mgr.z for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:41:33.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:33 smithi046 ceph-mon[30951]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:34.028 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:33 smithi196 ceph-mon[34587]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:34.029 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:33 smithi196 podman[36376]: 2024-09-12 02:41:33.668698291 +0000 UTC m=+0.016066021 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:41:34.029 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:33 smithi196 podman[36376]: 2024-09-12 02:41:33.808183764 +0000 UTC m=+0.155551482 container create ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:41:34.029 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 podman[36376]: 2024-09-12 02:41:34.001982937 +0000 UTC m=+0.349350654 container init ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:41:34.029 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 podman[36376]: 2024-09-12 02:41:34.004704252 +0000 UTC m=+0.352071976 container start ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T02:41:34.029 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-mgr[36396]: -- 172.21.15.196:0/2460136031 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d6fb850e0 con 0x562d6fb62800 2024-09-12T02:41:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:33 smithi145 ceph-mon[34674]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:34.332 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 bash[36376]: ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b 2024-09-12T02:41:34.333 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 systemd[1]: Started Ceph mgr.z for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:41:34.333 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.123+0000 7f4c7e5d0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:41:34.333 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.185+0000 7f4c7e5d0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:41:34.333 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.330+0000 7f4c7e5d0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:41:34.682 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.396+0000 7f4c7e5d0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:41:34.682 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.542+0000 7f4c7e5d0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:41:34.682 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.681+0000 7f4c7e5d0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:41:34.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:41:34.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-09-12T02:41:35.042 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:35.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:35.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:35.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:34 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:34.990+0000 7f4c7e5d0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:41:35.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:35.050+0000 7f4c7e5d0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:41:35.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:35.232+0000 7f4c7e5d0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:41:35.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 -- 172.21.15.46:0/1107676385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100af0 msgr2=0x7f39bc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 --2- 172.21.15.46:0/1107676385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100af0 0x7f39bc100ef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009900 tx=0x7f39b802a410 comp rx=0 tx=0).stop 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 -- 172.21.15.46:0/1107676385 shutdown_connections 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 --2- 172.21.15.46:0/1107676385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc1026b0 0x7f39bc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 --2- 172.21.15.46:0/1107676385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc101cf0 0x7f39bc102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 --2- 172.21.15.46:0/1107676385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100af0 0x7f39bc100ef0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 -- 172.21.15.46:0/1107676385 >> 172.21.15.46:0/1107676385 conn(0x7f39bc0fc2a0 msgr2=0x7f39bc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 -- 172.21.15.46:0/1107676385 shutdown_connections 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 -- 172.21.15.46:0/1107676385 wait complete. 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.450+0000 7f39c34e0640 1 Processor -- start 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 -- start start 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 0x7f39bc10b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc1026b0 0x7f39bc10a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10d750 con 0x7f39bc101cf0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10d5d0 con 0x7f39bc100af0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c34e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10d8d0 con 0x7f39bc1026b0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33266/0 (socket says 172.21.15.46:33266) 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 -- 172.21.15.46:0/2696114159 learned_addr learned my addr 172.21.15.46:0/2696114159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c1a7d640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc1026b0 0x7f39bc10a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c127c640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 0x7f39bc10b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 -- 172.21.15.46:0/2696114159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc1026b0 msgr2=0x7f39bc10a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc1026b0 0x7f39bc10a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 -- 172.21.15.46:0/2696114159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 msgr2=0x7f39bc10b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 0x7f39bc10b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 -- 172.21.15.46:0/2696114159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc10a8c0 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c127c640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 0x7f39bc10b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.451+0000 7f39c0a7b640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00d950 tx=0x7f39ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.452+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac017030 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.452+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39ac004020 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.452+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a1fb0 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.452+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0047a0 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.452+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a24b0 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f39ac004940 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39b27fc640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 0x7f39a803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc107f90 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f39ac04bde0 con 0x7f39bc101cf0 2024-09-12T02:41:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39c127c640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 0x7f39a803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:41:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.453+0000 7f39c127c640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 0x7f39a803b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39b8000c00 tx=0x7f39b8032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:41:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.458+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39ac00fcb0 con 0x7f39bc101cf0 2024-09-12T02:41:35.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:35.643+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f39bc0626f0 con 0x7f39a8039290 2024-09-12T02:41:36.284 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:36.284 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: Deploying daemon mgr.y on smithi145 2024-09-12T02:41:36.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:36.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:41:36.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:41:36.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: Deploying daemon mgr.y on smithi145 2024-09-12T02:41:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:41:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:41:36.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:36.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:36.101+0000 7f4c7e5d0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:41:36.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:36.168+0000 7f4c7e5d0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:41:36.477 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:36.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: Deploying daemon mgr.y on smithi145 2024-09-12T02:41:36.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:41:36.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:41:36.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:41:36.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:36.731 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:36.729+0000 7f58b84c5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:41:37.110 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:36.798+0000 7f58b84c5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:41:37.110 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:36 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:36.974+0000 7f58b84c5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:41:37.195 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:36 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:36.902+0000 7f4c7e5d0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:41:37.195 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.049+0000 7f4c7e5d0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:41:37.195 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.194+0000 7f4c7e5d0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:41:37.379 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:37.124+0000 7f58b84c5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:41:37.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.255+0000 7f4c7e5d0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:41:37.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.419+0000 7f4c7e5d0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:41:37.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:37.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:37.456+0000 7f58b84c5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:37.518+0000 7f58b84c5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:41:37.714 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:37 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:37.712+0000 7f58b84c5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:41:37.802 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.488+0000 7f4c7e5d0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.550+0000 7f4c7e5d0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.712+0000 7f4c7e5d0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:41:37.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.801+0000 7f4c7e5d0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:41:38.073 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.869+0000 7f4c7e5d0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:41:38.073 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:37 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:37.954+0000 7f4c7e5d0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:41:38.073 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:38.013+0000 7f4c7e5d0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:41:38.073 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:38.072+0000 7f4c7e5d0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:38.597+0000 7f58b84c5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:38.665+0000 7f58b84c5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: Reconfiguring daemon mgr.x on smithi046 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]': finished 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:41:38.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:38 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2702291211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:41:38.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: Reconfiguring daemon mgr.x on smithi046 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]: dispatch 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]': finished 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:41:38.955 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:38 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2702291211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:41:39.220 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:41:38.853+0000 7f4c7e5d0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:41:39.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:39.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: Reconfiguring daemon mgr.x on smithi046 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]: dispatch 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4054606190' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aec0a35b-0993-4783-ae91-001115bfc027"}]': finished 2024-09-12T02:41:39.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:41:39.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:41:39.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:38 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2702291211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:41:39.636 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.377+0000 7f58b84c5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:41:39.636 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.493+0000 7f58b84c5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:41:39.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:39.729+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f39ac003b10 con 0x7f39bc101cf0 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: Standby manager daemon z started 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:41:39.916 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:39.917 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.635+0000 7f58b84c5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:41:39.917 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.696+0000 7f58b84c5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:41:39.917 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.847+0000 7f58b84c5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:41:39.917 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.914+0000 7f58b84c5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:41:40.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: Standby manager daemon z started 2024-09-12T02:41:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:41:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:41:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:39 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:40.201 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:39 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:39.975+0000 7f58b84c5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:41:40.201 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.133+0000 7f58b84c5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:41:40.201 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.199+0000 7f58b84c5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: Standby manager daemon z started 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:41:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:39 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/1747478996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:40.465 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.265+0000 7f58b84c5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:41:40.465 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.346+0000 7f58b84c5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:41:40.465 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.404+0000 7f58b84c5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:41:40.465 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:40.463+0000 7f58b84c5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:41:41.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:40 smithi046 ceph-mon[30951]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:41:41.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:41:41.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-mon[34674]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:41:41.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:41:41.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:40 smithi196 ceph-mon[34587]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:41:41.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:41:41.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:41:41.253+0000 7f58b84c5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: Standby manager daemon y started 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:41:42.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:41 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: Standby manager daemon y started 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:41:42.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:41 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:42.211+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f39ac004c30 con 0x7f39bc101cf0 2024-09-12T02:41:42.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: Standby manager daemon y started 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:41:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:41 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/2713789795' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:41:43.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:43 smithi196 ceph-mon[34587]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:43.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:43 smithi196 ceph-mon[34587]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T02:41:43.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:41:43.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:43 smithi046 ceph-mon[30951]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:43.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:43 smithi046 ceph-mon[30951]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T02:41:43.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:41:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:43 smithi145 ceph-mon[34674]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:43 smithi145 ceph-mon[34674]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-12T02:41:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:41:46.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:46.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:46.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:46.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:46.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:46.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.581 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:46.582 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:48.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:48 smithi046 ceph-mon[30951]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:48.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:41:48.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:48 smithi196 ceph-mon[34587]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:41:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:48 smithi145 ceph-mon[34674]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:41:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:49.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:49 smithi046 ceph-mon[30951]: Deploying daemon osd.0 on smithi046 2024-09-12T02:41:49.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:49 smithi196 ceph-mon[34587]: Deploying daemon osd.0 on smithi046 2024-09-12T02:41:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:49 smithi145 ceph-mon[34674]: Deploying daemon osd.0 on smithi046 2024-09-12T02:41:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:50 smithi196 ceph-mon[34587]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:50.481 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:50 smithi046 ceph-mon[30951]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:50 smithi145 ceph-mon[34674]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:52 smithi196 ceph-mon[34587]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:52.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:52 smithi046 ceph-mon[30951]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:52 smithi145 ceph-mon[34674]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:54.435 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:54 smithi046 ceph-mon[30951]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:54.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:54.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:54.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:54 smithi196 ceph-mon[34587]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:54.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:54.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:54 smithi145 ceph-mon[34674]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:54.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:54.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:54.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:56.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:56 smithi046 ceph-mon[30951]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:56 smithi196 ceph-mon[34587]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:56 smithi145 ceph-mon[34674]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:57.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:57.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:57.727 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:41:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.000+0000 7f39b27fc640 1 -- 172.21.15.46:0/2696114159 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39bc0626f0 con 0x7f39a8039290 2024-09-12T02:41:58.002 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-09-12T02:41:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 msgr2=0x7f39a803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:58.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 0x7f39a803b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39b8000c00 tx=0x7f39b8032c40 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 msgr2=0x7f39bc109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00d950 tx=0x7f39ac00de10 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 shutdown_connections 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39a8039290 0x7f39a803b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc1026b0 0x7f39bc10a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc101cf0 0x7f39bc109ac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 --2- 172.21.15.46:0/2696114159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39bc100af0 0x7f39bc10b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 >> 172.21.15.46:0/2696114159 conn(0x7f39bc0fc2a0 msgr2=0x7f39bc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 shutdown_connections 2024-09-12T02:41:58.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:41:58.003+0000 7f39c34e0640 1 -- 172.21.15.46:0/2696114159 wait complete. 2024-09-12T02:41:58.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:58.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:41:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.626 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T02:41:58.629 INFO:tasks.cephadm:Deploying osd.1 on smithi145 with /dev/vg_nvme/lv_4... 2024-09-12T02:41:58.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:41:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:41:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:41:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:41:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:41:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:42:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:00 smithi046 ceph-mon[30951]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:00 smithi046 ceph-mon[30951]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:42:00.033 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:41:59 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T02:41:59.881+0000 7fc7bed1b640 -1 osd.0 0 log_to_monitors true 2024-09-12T02:42:00.183 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:00 smithi145 ceph-mon[34674]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:00.184 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:00 smithi145 ceph-mon[34674]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:42:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:00 smithi196 ceph-mon[34587]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:00 smithi196 ceph-mon[34587]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:42:01.113 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T02:42:01.006+0000 7fc7bac9e640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T02:42:01.113 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T02:42:01.014+0000 7fc7b6ac6640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:42:01.113 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-mon[30951]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:42:01.114 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-mon[30951]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:42:01.114 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-mon[30951]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:42:01.114 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:01.114 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:01 smithi046 ceph-mon[30951]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:42:01.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:01 smithi145 ceph-mon[34674]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:42:01.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:01 smithi145 ceph-mon[34674]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:42:01.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:01 smithi145 ceph-mon[34674]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:42:01.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:01.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:01 smithi145 ceph-mon[34674]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:42:01.392 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:42:01.423 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-09-12T02:42:01.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:01 smithi196 ceph-mon[34587]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:42:01.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:01 smithi196 ceph-mon[34587]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:42:01.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:01 smithi196 ceph-mon[34587]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:42:01.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:01.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:01 smithi196 ceph-mon[34587]: from='osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:42:01.676 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:42:02.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.137+0000 7f858435b640 1 -- 172.21.15.145:0/4030417785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 msgr2=0x7f857c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.137+0000 7f858435b640 1 --2- 172.21.15.145:0/4030417785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f856c009900 tx=0x7f856c02a410 comp rx=0 tx=0).stop 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- 172.21.15.145:0/4030417785 shutdown_connections 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- 172.21.15.145:0/4030417785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 0x7f857c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- 172.21.15.145:0/4030417785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- 172.21.15.145:0/4030417785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 0x7f857c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- 172.21.15.145:0/4030417785 >> 172.21.15.145:0/4030417785 conn(0x7f857c0fc280 msgr2=0x7f857c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- 172.21.15.145:0/4030417785 shutdown_connections 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- 172.21.15.145:0/4030417785 wait complete. 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 Processor -- start 2024-09-12T02:42:02.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- start start 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 0x7f857c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 0x7f857c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10c660 con 0x7f857c100ad0 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10c4e0 con 0x7f857c101cd0 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f858435b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10c7e0 con 0x7f857c102690 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.138+0000 7f85818f6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36506/0 (socket says 172.21.15.145:36506) 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 -- 172.21.15.145:0/4135692744 learned_addr learned my addr 172.21.15.145:0/4135692744 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85820f7640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 0x7f857c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85828f8640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 0x7f857c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 -- 172.21.15.145:0/4135692744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 msgr2=0x7f857c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 0x7f857c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 -- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 msgr2=0x7f857c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 0x7f857c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 -- 172.21.15.145:0/4135692744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c19fad0 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85818f6640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f856c004770 tx=0x7f856c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c03f030 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c19fd60 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f856c003cb0 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c035070 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.139+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c1a0230 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.141+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c1a0460 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.141+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f856c042030 con 0x7f857c101cd0 2024-09-12T02:42:02.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.142+0000 7f85737fe640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 0x7f85680a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:02.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.142+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f856c0edaf0 con 0x7f857c101cd0 2024-09-12T02:42:02.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.142+0000 7f85820f7640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 0x7f85680a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:02.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.143+0000 7f85820f7640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 0x7f85680a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f857800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:02.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.146+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f856c0bc4c0 con 0x7f857c101cd0 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:02.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:02.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:02.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:02.375+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f857c06a730 con 0x7f85680a2ba0 2024-09-12T02:42:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:02.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi046 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 17092M 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] boot 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:03.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:03.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:42:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:42:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi046 2024-09-12T02:42:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 17092M 2024-09-12T02:42:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] boot 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:03.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:42:03.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:42:03.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi046 2024-09-12T02:42:03.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 17092M 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: osd.0 [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] boot 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:03.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:04.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:04 smithi046 ceph-mon[30951]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:04 smithi046 ceph-mon[30951]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:04 smithi046 ceph-mon[30951]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:42:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:04 smithi145 ceph-mon[34674]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:04 smithi145 ceph-mon[34674]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:04 smithi145 ceph-mon[34674]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:42:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:04 smithi196 ceph-mon[34587]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:42:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:04 smithi196 ceph-mon[34587]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:04 smithi196 ceph-mon[34587]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:42:05.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:05 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/2478942824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:05 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:05 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]': finished 2024-09-12T02:42:05.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:05 smithi046 ceph-mon[30951]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:42:05.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:05.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:05 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/2478942824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:05 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:05 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]': finished 2024-09-12T02:42:05.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:05 smithi145 ceph-mon[34674]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:42:05.285 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:05.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:05 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/2478942824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:05 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]: dispatch 2024-09-12T02:42:05.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:05 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "701e099d-ad1c-45cd-b9e3-8164c0daa034"}]': finished 2024-09-12T02:42:05.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:05 smithi196 ceph-mon[34587]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:42:05.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:06.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:06 smithi046 ceph-mon[30951]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:06.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:06 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/1720849057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:06.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:06 smithi145 ceph-mon[34674]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:06.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:06 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/1720849057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:06 smithi196 ceph-mon[34587]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:06 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/1720849057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:08.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:08 smithi046 ceph-mon[30951]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:08.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:08 smithi145 ceph-mon[34674]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:08.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:08 smithi196 ceph-mon[34587]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:10.114 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:10 smithi145 ceph-mon[34674]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:10.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:10 smithi046 ceph-mon[30951]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:10 smithi196 ceph-mon[34587]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:12.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:12 smithi145 ceph-mon[34674]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:12 smithi196 ceph-mon[34587]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:12.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:12 smithi046 ceph-mon[30951]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:14.274 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:14 smithi145 ceph-mon[34674]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:14.275 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:42:14.275 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:14 smithi196 ceph-mon[34587]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:42:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:14 smithi046 ceph-mon[30951]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:14.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:42:14.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:15.152 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:15 smithi145 ceph-mon[34674]: Deploying daemon osd.1 on smithi145 2024-09-12T02:42:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:15 smithi196 ceph-mon[34587]: Deploying daemon osd.1 on smithi145 2024-09-12T02:42:15.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:15 smithi046 ceph-mon[30951]: Deploying daemon osd.1 on smithi145 2024-09-12T02:42:16.299 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:16 smithi145 ceph-mon[34674]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:16.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:16 smithi196 ceph-mon[34587]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:16 smithi046 ceph-mon[30951]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:18.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:18 smithi145 ceph-mon[34674]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:18 smithi196 ceph-mon[34587]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:18 smithi046 ceph-mon[30951]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:20.330 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:20 smithi145 ceph-mon[34674]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:20.330 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:20.330 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:20.330 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:20.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:20 smithi196 ceph-mon[34587]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:20 smithi046 ceph-mon[30951]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:22.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:22.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:22.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:22.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:23.783 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 1 on host 'smithi145' 2024-09-12T02:42:23.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.782+0000 7f85737fe640 1 -- 172.21.15.145:0/4135692744 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f857c06a730 con 0x7f85680a2ba0 2024-09-12T02:42:23.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 msgr2=0x7f85680a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:23.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 0x7f85680a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f857800b410 comp rx=0 tx=0).stop 2024-09-12T02:42:23.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 msgr2=0x7f857c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:23.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f856c004770 tx=0x7f856c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 shutdown_connections 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85680a2ba0 0x7f85680a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f857c102690 0x7f857c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f857c101cd0 0x7f857c1983e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 --2- 172.21.15.145:0/4135692744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c100ad0 0x7f857c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 >> 172.21.15.145:0/4135692744 conn(0x7f857c0fc280 msgr2=0x7f857c06a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 shutdown_connections 2024-09-12T02:42:23.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:42:23.784+0000 7f858435b640 1 -- 172.21.15.145:0/4135692744 wait complete. 2024-09-12T02:42:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:24 smithi145 ceph-mon[34674]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:24 smithi196 ceph-mon[34587]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:24.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:24.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.532 DEBUG:teuthology.orchestra.run.smithi145:osd.1> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T02:42:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:24 smithi046 ceph-mon[30951]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:24.534 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-09-12T02:42:24.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:42:24.786 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:42:25.815 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:25 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T02:42:25.551+0000 7f59bf7fc640 -1 osd.1 0 log_to_monitors true 2024-09-12T02:42:26.090 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:26 smithi145 ceph-mon[34674]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:26.091 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:26 smithi145 ceph-mon[34674]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:26.091 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:26 smithi145 ceph-mon[34674]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:26.127 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:26 smithi196 ceph-mon[34587]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:26.127 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:26 smithi196 ceph-mon[34587]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:26.127 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:26 smithi196 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:26.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:26 smithi046 ceph-mon[30951]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:26.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:26 smithi046 ceph-mon[30951]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:26.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:26 smithi046 ceph-mon[30951]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:42:27.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:42:27.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:42:27.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:27.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.189 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:27 smithi196 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:42:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:42:27.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.345 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-mon[34674]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:42:27.462 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: from='osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:42:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:27 smithi046 ceph-mon[30951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:42:27.610 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T02:42:27.512+0000 7f59bb77f640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T02:42:27.610 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:27 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T02:42:27.518+0000 7f59b359f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 -- 172.21.15.196:0/3714767030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0101cd0 msgr2=0x7f65b0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/3714767030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0101cd0 0x7f65b0102150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009900 tx=0x7f65a402a410 comp rx=0 tx=0).stop 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 -- 172.21.15.196:0/3714767030 shutdown_connections 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/3714767030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0102690 0x7f65b0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/3714767030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0101cd0 0x7f65b0102150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/3714767030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 0x7f65b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.908+0000 7f65b5ad3640 1 -- 172.21.15.196:0/3714767030 >> 172.21.15.196:0/3714767030 conn(0x7f65b00fc280 msgr2=0x7f65b00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.909+0000 7f65b5ad3640 1 -- 172.21.15.196:0/3714767030 shutdown_connections 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.909+0000 7f65b5ad3640 1 -- 172.21.15.196:0/3714767030 wait complete. 2024-09-12T02:42:27.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.909+0000 7f65b5ad3640 1 Processor -- start 2024-09-12T02:42:27.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.909+0000 7f65b5ad3640 1 -- start start 2024-09-12T02:42:27.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.909+0000 7f65b5ad3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 0x7f65b0104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:27.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 0x7f65b01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c700 con 0x7f65b0100ad0 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c580 con 0x7f65b0101cd0 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010c880 con 0x7f65b0102690 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56410/0 (socket says 172.21.15.196:56410) 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 -- 172.21.15.196:0/2681255376 learned_addr learned my addr 172.21.15.196:0/2681255376 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65af7fe640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 0x7f65b0104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65aeffd640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 0x7f65b01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 -- 172.21.15.196:0/2681255376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 msgr2=0x7f65b01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 0x7f65b01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:27.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 -- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 msgr2=0x7f65b0104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 0x7f65b0104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 -- 172.21.15.196:0/2681255376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b0103eb0 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65aeffd640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 0x7f65b01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65affff640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65a800eaa0 tx=0x7f65a800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004d20 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a8005020 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80116b0 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.910+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01a2060 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.911+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01a25a0 con 0x7f65b0102690 2024-09-12T02:42:27.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.912+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b01a2830 con 0x7f65b0102690 2024-09-12T02:42:27.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.913+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f65a80051d0 con 0x7f65b0102690 2024-09-12T02:42:27.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.913+0000 7f65acff9640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 0x7f65980a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:27.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.914+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f65a80cdc80 con 0x7f65b0102690 2024-09-12T02:42:27.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.914+0000 7f65af7fe640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 0x7f65980a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:27.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.914+0000 7f65af7fe640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 0x7f65980a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65b0101b30 tx=0x7f65a0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:27.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:27.917+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65a8016030 con 0x7f65b0102690 2024-09-12T02:42:28.102 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:28.102 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='osd.1 ' entity='osd.1' 2024-09-12T02:42:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:28.101+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f65b01a2a00 con 0x7f65980a2af0 2024-09-12T02:42:28.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:28.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:42:28.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='osd.1 ' entity='osd.1' 2024-09-12T02:42:28.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:28.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:28.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:28.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='osd.1 ' entity='osd.1' 2024-09-12T02:42:28.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:29.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:42:29.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:42:29.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi145 2024-09-12T02:42:29.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-12T02:42:29.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] boot 2024-09-12T02:42:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:42:29.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:29.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:42:29.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:42:29.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi145 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] boot 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:42:29.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi145 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: osd.1 [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] boot 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:42:29.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:42:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:30 smithi145 ceph-mon[34674]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:30 smithi145 ceph-mon[34674]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:42:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:30 smithi196 ceph-mon[34587]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:30 smithi196 ceph-mon[34587]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:42:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:30 smithi046 ceph-mon[30951]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:42:30.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:30 smithi046 ceph-mon[30951]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/3605487443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]': finished 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:31.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:31 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/2875718404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/3605487443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]': finished 2024-09-12T02:42:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:42:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:31.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:31 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/2875718404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/3605487443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]: dispatch 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab049462-5f42-4b64-8536-edca415271ff"}]': finished 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:31 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/2875718404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:32.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:32 smithi145 ceph-mon[34674]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:32.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:32 smithi196 ceph-mon[34587]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:32.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:32 smithi046 ceph-mon[30951]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:34.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:34 smithi145 ceph-mon[34674]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:34 smithi196 ceph-mon[34587]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:34.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:34 smithi046 ceph-mon[30951]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:36 smithi145 ceph-mon[34674]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:36.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:36 smithi196 ceph-mon[34587]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:36 smithi046 ceph-mon[30951]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:38.346 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:38 smithi196 ceph-mon[34587]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:38 smithi145 ceph-mon[34674]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:38.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:38 smithi046 ceph-mon[30951]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:39.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:42:39.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:42:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:39.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:42:39.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:40 smithi145 ceph-mon[34674]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:40 smithi145 ceph-mon[34674]: Deploying daemon osd.2 on smithi196 2024-09-12T02:42:40.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:40 smithi196 ceph-mon[34587]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:40.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:40 smithi196 ceph-mon[34587]: Deploying daemon osd.2 on smithi196 2024-09-12T02:42:40.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:40 smithi046 ceph-mon[30951]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:40.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:40 smithi046 ceph-mon[30951]: Deploying daemon osd.2 on smithi196 2024-09-12T02:42:41.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:41 smithi145 ceph-mon[34674]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:41.389 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:41 smithi196 ceph-mon[34587]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:41.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:41 smithi046 ceph-mon[30951]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:43.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:43 smithi046 ceph-mon[30951]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:43 smithi145 ceph-mon[34674]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:43.703 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:43 smithi196 ceph-mon[34587]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:45.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:45 smithi046 ceph-mon[30951]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:45.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:45.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:45.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:45 smithi145 ceph-mon[34674]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:45.673 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:45 smithi196 ceph-mon[34587]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:45.673 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:45.673 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:45.673 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:47.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:47 smithi046 ceph-mon[30951]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:47.579 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:47 smithi196 ceph-mon[34587]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:47.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:47 smithi145 ceph-mon[34674]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:48.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.747+0000 7f65acff9640 1 -- 172.21.15.196:0/2681255376 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65b01a2a00 con 0x7f65980a2af0 2024-09-12T02:42:48.749 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 msgr2=0x7f65980a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 0x7f65980a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65b0101b30 tx=0x7f65a0009110 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 msgr2=0x7f65b01035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65a800eaa0 tx=0x7f65a800eea0 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 shutdown_connections 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f65980a2af0 0x7f65980a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b0102690 0x7f65b01035f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65b0101cd0 0x7f65b01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 --2- 172.21.15.196:0/2681255376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b0100ad0 0x7f65b0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 >> 172.21.15.196:0/2681255376 conn(0x7f65b00fc280 msgr2=0x7f65b006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:42:48.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 shutdown_connections 2024-09-12T02:42:48.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:42:48.750+0000 7f65b5ad3640 1 -- 172.21.15.196:0/2681255376 wait complete. 2024-09-12T02:42:48.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:48.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:48.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:49.363 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T02:42:49.365 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-09-12T02:42:49.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:42:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:42:50.022 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:49 smithi046 ceph-mon[30951]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:49 smithi196 ceph-mon[34587]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:50.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:49 smithi145 ceph-mon[34674]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:51.789 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:51 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T02:42:51.533+0000 7fef8bd5a640 -1 osd.2 0 log_to_monitors true 2024-09-12T02:42:52.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:51 smithi196 ceph-mon[34587]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:52.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:51 smithi196 ceph-mon[34587]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:51 smithi196 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.180 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:51 smithi046 ceph-mon[30951]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:52.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:51 smithi046 ceph-mon[30951]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:51 smithi046 ceph-mon[30951]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:51 smithi145 ceph-mon[34674]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:52.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:51 smithi145 ceph-mon[34674]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:51 smithi145 ceph-mon[34674]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:42:52.420 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:42:52.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-09-12T02:42:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi196 2024-09-12T02:42:53.137 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:53.138 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:52 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.146+0000 7f168b16d640 1 -- 172.21.15.46:0/224565142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684100ad0 msgr2=0x7f1684100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.146+0000 7f168b16d640 1 --2- 172.21.15.46:0/224565142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684100ad0 0x7f1684100ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a410 comp rx=0 tx=0).stop 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.146+0000 7f168b16d640 1 -- 172.21.15.46:0/224565142 shutdown_connections 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.146+0000 7f168b16d640 1 --2- 172.21.15.46:0/224565142 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684102690 0x7f1684108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 --2- 172.21.15.46:0/224565142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 0x7f1684102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 --2- 172.21.15.46:0/224565142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684100ad0 0x7f1684100ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 -- 172.21.15.46:0/224565142 >> 172.21.15.46:0/224565142 conn(0x7f16840fc280 msgr2=0x7f16840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 -- 172.21.15.46:0/224565142 shutdown_connections 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 -- 172.21.15.46:0/224565142 wait complete. 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 Processor -- start 2024-09-12T02:42:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.147+0000 7f168b16d640 1 -- start start 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684100ad0 0x7f168419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 0x7f168419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f168410c600 con 0x7f1684102690 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f168410c480 con 0x7f1684100ad0 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168b16d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f168410c780 con 0x7f1684101cd0 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f1683fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 0x7f168419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41568/0 (socket says 172.21.15.46:41568) 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 -- 172.21.15.46:0/4216525621 learned_addr learned my addr 172.21.15.46:0/4216525621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f1688f09640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684100ad0 0x7f168419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 -- 172.21.15.46:0/4216525621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 msgr2=0x7f168419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 0x7f168419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 -- 172.21.15.46:0/4216525621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684100ad0 msgr2=0x7f168419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684100ad0 0x7f168419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 -- 172.21.15.46:0/4216525621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16841a1cc0 con 0x7f1684102690 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.148+0000 7f168970a640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f167c00eaa0 tx=0x7f167c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.149+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00cdb0 con 0x7f1684102690 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.149+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f167c005020 con 0x7f1684102690 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.149+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16841a1fb0 con 0x7f1684102690 2024-09-12T02:42:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.149+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c0116d0 con 0x7f1684102690 2024-09-12T02:42:53.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.149+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16841a2410 con 0x7f1684102690 2024-09-12T02:42:53.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.150+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16841a26d0 con 0x7f1684102690 2024-09-12T02:42:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.151+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f167c0051d0 con 0x7f1684102690 2024-09-12T02:42:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.152+0000 7f1681ffb640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 0x7f16740a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:42:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.152+0000 7f1688f09640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 0x7f16740a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:42:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.152+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f167c0cda80 con 0x7f1684102690 2024-09-12T02:42:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.152+0000 7f1688f09640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 0x7f16740a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1678004770 tx=0x7f1678000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:42:53.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.156+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f167c016030 con 0x7f1684102690 2024-09-12T02:42:53.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:42:53.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi196 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:53.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:52 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:53.343+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f16841a28a0 con 0x7f16740a2ba0 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi196 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:42:53.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:52 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:53.720 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T02:42:53.302+0000 7fef87cdd640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T02:42:53.720 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T02:42:53.307+0000 7fef7fafd640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='osd.2 ' entity='osd.2' 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:54.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:53 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:54.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='osd.2 ' entity='osd.2' 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:53 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='osd.2 ' entity='osd.2' 2024-09-12T02:42:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:42:54.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:42:54.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:53 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] boot 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:42:55.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:55.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] boot 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:42:55.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:55.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:42:55.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:42:55.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:42:55.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:55.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: osd.2 [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] boot 2024-09-12T02:42:55.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:42:55.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:42:56.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:56.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:42:56.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]: dispatch 2024-09-12T02:42:56.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]': finished 2024-09-12T02:42:56.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:42:56.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:56.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:42:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]: dispatch 2024-09-12T02:42:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]': finished 2024-09-12T02:42:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:42:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:56.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:42:56.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:42:56.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]: dispatch 2024-09-12T02:42:56.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3764779762' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ad0adf4-ee5f-4d68-8b92-27397454b3ee"}]': finished 2024-09-12T02:42:56.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:42:56.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:57.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:57.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:56 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1807467346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:57.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:56 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1807467346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:57.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:57.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:56 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1807467346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:42:58.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:57 smithi196 ceph-mon[34587]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:42:58.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:58.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:57 smithi196 ceph-mon[34587]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:42:58.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:58.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:58.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:57 smithi046 ceph-mon[30951]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:42:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:57 smithi046 ceph-mon[30951]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:42:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:58.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:57 smithi145 ceph-mon[34674]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:42:58.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:58.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:57 smithi145 ceph-mon[34674]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:42:58.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:58.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:42:58.782 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:42:58.783 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:42:58.783 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50214]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:42:58.783 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50214]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:42:58.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:42:58.947+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f167c0980b0 con 0x7f1684102690 2024-09-12T02:42:59.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:59.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:58 smithi145 ceph-mon[34674]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:42:59.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:59.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:59.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:58 smithi196 ceph-mon[34587]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:42:59.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:59.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:42:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:58 smithi046 ceph-mon[30951]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:42:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:59.283 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50586]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:42:59.283 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50586]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:42:59.283 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50586]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:42:59.283 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 02:42:58 smithi046 sudo[50586]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:42:59.466 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:42:59.466 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:42:59.466 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43261]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:42:59.466 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43261]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:42:59.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43264]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:42:59.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:42:59.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43264]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:42:59.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 02:42:59 smithi145 sudo[43264]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:42:59.995 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:42:59.995 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:42:59.995 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43310]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:42:59.995 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43310]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:42:59.995 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:42:59.995 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:42:59.995 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:42:59.995 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:42:59.996 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:42:59.996 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:42:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:42:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:00.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:00.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:43:00.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:00.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:43:00.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:00.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:42:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:00.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:43:00.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:00.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43313]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:00.471 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 02:42:59 smithi196 sudo[43313]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:00.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[50968]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:43:00.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[50968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:00.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[50968]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:00.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[50968]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:01.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[51061]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:43:01.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[51061]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:01.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[51061]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:01.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:00 smithi046 sudo[51061]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:01.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43316]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:43:01.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43316]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:01.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43316]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:01.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43316]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43319]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 sudo[43319]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:01.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:01.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:01.919 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:43:01.919 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:01.920 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43267]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:01.920 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43267]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43270]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 sudo[43270]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:02.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:02.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:02 smithi196 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:02.743 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:02 smithi046 ceph-mon[30951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:43:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:43:02.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:43:02.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:02 smithi145 ceph-mon[34674]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:43:03.596 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:03 smithi046 ceph-mon[30951]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:03 smithi196 ceph-mon[34587]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:03 smithi145 ceph-mon[34674]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:43:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:43:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:04.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:43:04.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:05.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:05 smithi196 ceph-mon[34587]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-12T02:43:05.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:05 smithi196 ceph-mon[34587]: Deploying daemon osd.3 on smithi046 2024-09-12T02:43:05.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:05 smithi046 ceph-mon[30951]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-12T02:43:05.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:05 smithi046 ceph-mon[30951]: Deploying daemon osd.3 on smithi046 2024-09-12T02:43:05.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:05 smithi145 ceph-mon[34674]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-12T02:43:05.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:05 smithi145 ceph-mon[34674]: Deploying daemon osd.3 on smithi046 2024-09-12T02:43:07.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:07 smithi196 ceph-mon[34587]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-12T02:43:07.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:07 smithi046 ceph-mon[30951]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-12T02:43:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:07 smithi145 ceph-mon[34674]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-12T02:43:09.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:09 smithi196 ceph-mon[34587]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-12T02:43:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:09 smithi046 ceph-mon[30951]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-12T02:43:09.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:09 smithi145 ceph-mon[34674]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-12T02:43:11.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:11 smithi046 ceph-mon[30951]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T02:43:11.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:11.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:11.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:11 smithi196 ceph-mon[34587]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T02:43:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:11.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:11.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:11 smithi145 ceph-mon[34674]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-12T02:43:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:13.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:13 smithi196 ceph-mon[34587]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:13.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:13 smithi046 ceph-mon[30951]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:13 smithi145 ceph-mon[34674]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.882+0000 7f1681ffb640 1 -- 172.21.15.46:0/4216525621 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f16841a28a0 con 0x7f16740a2ba0 2024-09-12T02:43:14.884 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-09-12T02:43:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 msgr2=0x7f16740a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 0x7f16740a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1678004770 tx=0x7f1678000bc0 comp rx=0 tx=0).stop 2024-09-12T02:43:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 msgr2=0x7f16841a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f167c00eaa0 tx=0x7f167c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 shutdown_connections 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1684102690 0x7f16841a1650 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16740a2ba0 0x7f16740a5060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1684101cd0 0x7f168419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 --2- 172.21.15.46:0/4216525621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1684100ad0 0x7f168419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 >> 172.21.15.46:0/4216525621 conn(0x7f16840fc280 msgr2=0x7f168406a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 shutdown_connections 2024-09-12T02:43:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:43:14.885+0000 7f168b16d640 1 -- 172.21.15.46:0/4216525621 wait complete. 2024-09-12T02:43:14.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:14.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:14.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:14.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.518 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T02:43:15.521 INFO:tasks.cephadm:Deploying osd.4 on smithi145 with /dev/vg_nvme/lv_3... 2024-09-12T02:43:15.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:43:15.773 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:43:15.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:15 smithi046 ceph-mon[30951]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:15.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:15.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:15 smithi145 ceph-mon[34674]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:15 smithi196 ceph-mon[34587]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-12T02:43:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:16.934 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 02:43:16 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T02:43:16.836+0000 7f1289e1f640 -1 osd.3 0 log_to_monitors true 2024-09-12T02:43:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:17 smithi145 ceph-mon[34674]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:17 smithi145 ceph-mon[34674]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:43:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:17 smithi196 ceph-mon[34587]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:17 smithi196 ceph-mon[34587]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:43:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:17 smithi046 ceph-mon[30951]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:17 smithi046 ceph-mon[30951]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:43:18.376 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:43:18.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-09-12T02:43:18.662 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:43:18.696 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:18.697 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:18 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.782 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T02:43:18.552+0000 7f1285da2640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T02:43:18.783 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T02:43:18.558+0000 7f1281bca640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:18 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:18.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:18 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 -- 172.21.15.145:0/3000519817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0102fb0 msgr2=0x7f85d0103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 --2- 172.21.15.145:0/3000519817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0102fb0 0x7f85d0103430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009930 tx=0x7f85c402a420 comp rx=0 tx=0).stop 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 -- 172.21.15.145:0/3000519817 shutdown_connections 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 --2- 172.21.15.145:0/3000519817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0103970 0x7f85d010a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 --2- 172.21.15.145:0/3000519817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0102fb0 0x7f85d0103430 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 --2- 172.21.15.145:0/3000519817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 0x7f85d0100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:19.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 -- 172.21.15.145:0/3000519817 >> 172.21.15.145:0/3000519817 conn(0x7f85d00fba80 msgr2=0x7f85d00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.137+0000 7f85d562f640 1 -- 172.21.15.145:0/3000519817 shutdown_connections 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 -- 172.21.15.145:0/3000519817 wait complete. 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 Processor -- start 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 -- start start 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 0x7f85d0102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0102fb0 0x7f85d0100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:19.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d010d730 con 0x7f85d00fff30 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85d010d5b0 con 0x7f85d0103970 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.138+0000 7f85d562f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85d010d8b0 con 0x7f85d0102fb0 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36928/0 (socket says 172.21.15.145:36928) 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 -- 172.21.15.145:0/1739571652 learned_addr learned my addr 172.21.15.145:0/1739571652 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85ce7fc640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0102fb0 0x7f85d0100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85ceffd640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 0x7f85d0102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:19.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 -- 172.21.15.145:0/1739571652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0102fb0 msgr2=0x7f85d0100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0102fb0 0x7f85d0100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 -- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 msgr2=0x7f85d0102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 0x7f85d0102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 -- 172.21.15.145:0/1739571652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d0101d60 con 0x7f85d0103970 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85cf7fe640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85c800eb10 tx=0x7f85c800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8004b20 con 0x7f85d0103970 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d01a1fa0 con 0x7f85d0103970 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85c8004cc0 con 0x7f85d0103970 2024-09-12T02:43:19.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8010690 con 0x7f85d0103970 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.139+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01a2500 con 0x7f85d0103970 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.140+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0109250 con 0x7f85d0103970 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.142+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85c8005000 con 0x7f85d0103970 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.143+0000 7f85b7fff640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 0x7f85b80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.143+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3389+0+0 (secure 0 0 0) 0x7f85c80ce760 con 0x7f85d0103970 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.143+0000 7f85ceffd640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 0x7f85b80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.144+0000 7f85ceffd640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 0x7f85b80a5140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005d00 tx=0x7f85c0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:43:19.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.146+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85c809c960 con 0x7f85d0103970 2024-09-12T02:43:19.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:19.326+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f85d0062630 con 0x7f85b80a2c80 2024-09-12T02:43:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi046 2024-09-12T02:43:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 8546M 2024-09-12T02:43:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:43:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: osdmap e25: 4 total, 3 up, 4 in 2024-09-12T02:43:19.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:19.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:19.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi046 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 8546M 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: osdmap e25: 4 total, 3 up, 4 in 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:19.821 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:19.822 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi046 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 8546M 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: osdmap e25: 4 total, 3 up, 4 in 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:19.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:20.717 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:43:20.717 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:43:20.717 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:20.717 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.718 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] boot 2024-09-12T02:43:20.718 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:43:20.718 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.718 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:20 smithi145 ceph-mon[34674]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:43:20.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] boot 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:20 smithi046 ceph-mon[30951]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:43:20.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:43:20.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:43:20.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:20.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: osd.3 [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] boot 2024-09-12T02:43:20.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:43:20.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:43:20.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:20 smithi196 ceph-mon[34587]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:43:21.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:21 smithi196 ceph-mon[34587]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:21.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:21 smithi196 ceph-mon[34587]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:43:21.978 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:21 smithi145 ceph-mon[34674]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:21.978 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:21 smithi145 ceph-mon[34674]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:43:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:21 smithi046 ceph-mon[30951]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:43:22.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:21 smithi046 ceph-mon[30951]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:43:22.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/3390860686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]': finished 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:22.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:22 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/3887437586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/3390860686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]': finished 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:22 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/3887437586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/3390860686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]: dispatch 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62730f75-0e03-48fa-9cd3-d04a9e1d0046"}]': finished 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:43:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:23.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:22 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/3887437586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:24.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:23 smithi046 ceph-mon[30951]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:24.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:23 smithi145 ceph-mon[34674]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:23 smithi196 ceph-mon[34587]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:26.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:25 smithi046 ceph-mon[30951]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:43:26.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:25 smithi145 ceph-mon[34674]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:43:26.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:25 smithi196 ceph-mon[34587]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:43:28.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:27 smithi046 ceph-mon[30951]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:43:28.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:27 smithi145 ceph-mon[34674]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:43:28.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:27 smithi196 ceph-mon[34587]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:43:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:29 smithi046 ceph-mon[30951]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:43:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:29 smithi145 ceph-mon[34674]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:43:30.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:29 smithi196 ceph-mon[34587]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:43:31.100 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:43:31.100 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:31.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:43:31.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:31.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:43:31.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:32.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:31 smithi145 ceph-mon[34674]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:43:32.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:31 smithi145 ceph-mon[34674]: Deploying daemon osd.4 on smithi145 2024-09-12T02:43:32.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:31 smithi196 ceph-mon[34587]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:43:32.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:31 smithi196 ceph-mon[34587]: Deploying daemon osd.4 on smithi145 2024-09-12T02:43:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:31 smithi046 ceph-mon[30951]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:43:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:31 smithi046 ceph-mon[30951]: Deploying daemon osd.4 on smithi145 2024-09-12T02:43:34.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:33 smithi145 ceph-mon[34674]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T02:43:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:33 smithi196 ceph-mon[34587]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T02:43:34.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:33 smithi046 ceph-mon[30951]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T02:43:36.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:35 smithi145 ceph-mon[34674]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:43:36.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:35 smithi196 ceph-mon[34587]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:43:36.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:35 smithi046 ceph-mon[30951]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:43:38.085 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:37 smithi046 ceph-mon[30951]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:38.085 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:38.085 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:38.086 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:37 smithi145 ceph-mon[34674]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:37 smithi196 ceph-mon[34587]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:38.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:40.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:40.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:40.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:40.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:40.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:41.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.043+0000 7f85b7fff640 1 -- 172.21.15.145:0/1739571652 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f85d0062630 con 0x7f85b80a2c80 2024-09-12T02:43:41.044 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 4 on host 'smithi145' 2024-09-12T02:43:41.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 msgr2=0x7f85b80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:41.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 0x7f85b80a5140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005d00 tx=0x7f85c0005c50 comp rx=0 tx=0).stop 2024-09-12T02:43:41.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 msgr2=0x7f85d01014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:41.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85c800eb10 tx=0x7f85c800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 shutdown_connections 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85d0103970 0x7f85d01014a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f85b80a2c80 0x7f85b80a5140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d0102fb0 0x7f85d0100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 --2- 172.21.15.145:0/1739571652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d00fff30 0x7f85d0102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:41.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 >> 172.21.15.145:0/1739571652 conn(0x7f85d00fba80 msgr2=0x7f85d0106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:43:41.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 shutdown_connections 2024-09-12T02:43:41.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:43:41.045+0000 7f85d562f640 1 -- 172.21.15.145:0/1739571652 wait complete. 2024-09-12T02:43:41.840 DEBUG:teuthology.orchestra.run.smithi145:osd.4> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T02:43:41.842 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-09-12T02:43:41.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:43:42.097 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:43:42.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:41 smithi145 ceph-mon[34674]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:42.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:41 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:42.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:41 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:42.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:41 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:42.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:41 smithi196 ceph-mon[34587]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:42.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:41 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:42.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:41 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:42.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:41 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:42.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:41 smithi046 ceph-mon[30951]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:42.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:43:42.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:42.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:43.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 02:43:43 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T02:43:43.335+0000 7f0b6ab5b640 -1 osd.4 0 log_to_monitors true 2024-09-12T02:43:44.095 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:43 smithi196 ceph-mon[34587]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:44.096 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:43 smithi196 ceph-mon[34587]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.096 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:43 smithi196 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:43 smithi145 ceph-mon[34674]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:43 smithi145 ceph-mon[34674]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:43 smithi145 ceph-mon[34674]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:43 smithi046 ceph-mon[30951]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:44.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:43 smithi046 ceph-mon[30951]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:43 smithi046 ceph-mon[30951]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:43:44.793 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:43:44.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-09-12T02:43:45.082 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:45.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:43:45.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:43:45.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:45.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:45.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:43:45.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:45.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:43:45.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 -- 172.21.15.196:0/1029512823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf84104fd0 msgr2=0x7fdf841073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 --2- 172.21.15.196:0/1029512823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf84104fd0 0x7fdf841073c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009900 tx=0x7fdf7802a410 comp rx=0 tx=0).stop 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 -- 172.21.15.196:0/1029512823 shutdown_connections 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 --2- 172.21.15.196:0/1029512823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf84107900 0x7fdf84109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 --2- 172.21.15.196:0/1029512823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf84104fd0 0x7fdf841073c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 --2- 172.21.15.196:0/1029512823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf841026a0 0x7fdf84104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 -- 172.21.15.196:0/1029512823 >> 172.21.15.196:0/1029512823 conn(0x7fdf840fc2a0 msgr2=0x7fdf840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 -- 172.21.15.196:0/1029512823 shutdown_connections 2024-09-12T02:43:45.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 -- 172.21.15.196:0/1029512823 wait complete. 2024-09-12T02:43:45.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.577+0000 7fdf88d29640 1 Processor -- start 2024-09-12T02:43:45.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 -- start start 2024-09-12T02:43:45.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:45.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84104fd0 0x7fdf84198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:45.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf84107900 0x7fdf8419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d750 con 0x7fdf84104fd0 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d5d0 con 0x7fdf84107900 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d8d0 con 0x7fdf841026a0 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34750/0 (socket says 172.21.15.196:34750) 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 -- 172.21.15.196:0/2180071018 learned_addr learned my addr 172.21.15.196:0/2180071018 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf81d9b640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84104fd0 0x7fdf84198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 -- 172.21.15.196:0/2180071018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf84107900 msgr2=0x7fdf8419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf84107900 0x7fdf8419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 -- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84104fd0 msgr2=0x7fdf84198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84104fd0 0x7fdf84198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 -- 172.21.15.196:0/2180071018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8419fa40 con 0x7fdf841026a0 2024-09-12T02:43:45.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf8259c640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c920 tx=0x7fdf7400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:43:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74007c70 con 0x7fdf841026a0 2024-09-12T02:43:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.578+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8419fd30 con 0x7fdf841026a0 2024-09-12T02:43:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.579+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf74007e10 con 0x7fdf841026a0 2024-09-12T02:43:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.579+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74018820 con 0x7fdf841026a0 2024-09-12T02:43:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.579+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841a01c0 con 0x7fdf841026a0 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.580+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf840fdd90 con 0x7fdf841026a0 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.581+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdf740189c0 con 0x7fdf841026a0 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.581+0000 7fdf737fe640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 0x7fdf6c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.582+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7fdf740ce070 con 0x7fdf841026a0 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.582+0000 7fdf81d9b640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 0x7fdf6c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:43:45.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.582+0000 7fdf81d9b640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 0x7fdf6c0a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf7802a930 tx=0x7fdf78000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:43:45.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.586+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf74014030 con 0x7fdf841026a0 2024-09-12T02:43:45.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T02:43:45.212+0000 7f0b66ade640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T02:43:45.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T02:43:45.218+0000 7f0b5e8fe640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:43:45.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:43:45.761+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdf84062630 con 0x7fdf6c0a2bb0 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi145 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='osd.4 ' entity='osd.4' 2024-09-12T02:43:46.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:46.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:46.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:45 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:46.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:46.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi145 2024-09-12T02:43:46.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='osd.4 ' entity='osd.4' 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:45 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi145 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='osd.4 ' entity='osd.4' 2024-09-12T02:43:46.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:43:46.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:43:46.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:45 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] boot 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:43:47.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:47.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] boot 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:43:47.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: osd.4 [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] boot 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:43:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:43:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:47 smithi145 ceph-mon[34674]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:47 smithi145 ceph-mon[34674]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T02:43:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:47 smithi196 ceph-mon[34587]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:47 smithi196 ceph-mon[34587]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T02:43:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:47 smithi046 ceph-mon[30951]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:43:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:47 smithi046 ceph-mon[30951]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/395946864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]': finished 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:43:49.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:48 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/164278446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:49.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/395946864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]': finished 2024-09-12T02:43:49.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:43:49.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:43:49.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:48 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/164278446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/395946864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]: dispatch 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50ec1203-dfd3-402e-ae2f-e1c914c74c02"}]': finished 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:43:49.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:48 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/164278446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:43:50.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:49 smithi145 ceph-mon[34674]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:49 smithi196 ceph-mon[34587]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:50.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:49 smithi046 ceph-mon[30951]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:52.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:51 smithi145 ceph-mon[34674]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:52.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:51 smithi196 ceph-mon[34587]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:52.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:51 smithi046 ceph-mon[30951]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:53 smithi196 ceph-mon[34587]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:54.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:53 smithi145 ceph-mon[34674]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:54.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:53 smithi046 ceph-mon[30951]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:56.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:55 smithi196 ceph-mon[34587]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:56.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:55 smithi145 ceph-mon[34674]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:55 smithi046 ceph-mon[30951]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:57 smithi196 ceph-mon[34587]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:43:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:57 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:57.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:57 smithi196 ceph-mon[34587]: Deploying daemon osd.5 on smithi196 2024-09-12T02:43:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:57 smithi145 ceph-mon[34674]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:43:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:57 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:57 smithi145 ceph-mon[34674]: Deploying daemon osd.5 on smithi196 2024-09-12T02:43:58.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:57 smithi046 ceph-mon[30951]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:43:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:43:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:57 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:43:58.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:57 smithi046 ceph-mon[30951]: Deploying daemon osd.5 on smithi196 2024-09-12T02:44:00.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:43:59 smithi145 ceph-mon[34674]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:00.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:43:59 smithi196 ceph-mon[34587]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:00.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:43:59 smithi046 ceph-mon[30951]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:01.927 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:01 smithi196 ceph-mon[34587]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:01 smithi145 ceph-mon[34674]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:02.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:01 smithi046 ceph-mon[30951]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:03 smithi196 ceph-mon[34587]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:03 smithi145 ceph-mon[34674]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:03 smithi046 ceph-mon[30951]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:04.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:05 smithi145 ceph-mon[34674]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:06.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:05 smithi196 ceph-mon[34587]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:06.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:05 smithi046 ceph-mon[30951]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:07.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:07.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:07.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.410+0000 7fdf737fe640 1 -- 172.21.15.196:0/2180071018 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf84062630 con 0x7fdf6c0a2bb0 2024-09-12T02:44:07.412 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 msgr2=0x7fdf6c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 0x7fdf6c0a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf7802a930 tx=0x7fdf78000bc0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 msgr2=0x7fdf84197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c920 tx=0x7fdf7400cde0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 shutdown_connections 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf84107900 0x7fdf8419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf84104fd0 0x7fdf84198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf6c0a2bb0 0x7fdf6c0a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 --2- 172.21.15.196:0/2180071018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf841026a0 0x7fdf84197e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 >> 172.21.15.196:0/2180071018 conn(0x7fdf840fc2a0 msgr2=0x7fdf84103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 shutdown_connections 2024-09-12T02:44:07.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:44:07.414+0000 7fdf88d29640 1 -- 172.21.15.196:0/2180071018 wait complete. 2024-09-12T02:44:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.146 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T02:44:08.148 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-09-12T02:44:08.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:44:08.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:08 smithi196 ceph-mon[34587]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:08.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:08 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:08.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:08 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.377 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:08 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.423 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:44:08.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:08 smithi046 ceph-mon[30951]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:08.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:08 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:08.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:08 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:08 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:08 smithi145 ceph-mon[34674]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:08 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:08 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:08 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:09.576 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 02:44:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T02:44:09.292+0000 7fd1db2c5640 -1 osd.5 0 log_to_monitors true 2024-09-12T02:44:10.464 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:10 smithi046 ceph-mon[30951]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:10.464 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:10 smithi046 ceph-mon[30951]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:10.464 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:10 smithi046 ceph-mon[30951]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:10 smithi196 ceph-mon[34587]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:10 smithi196 ceph-mon[34587]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:10 smithi196 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:10 smithi145 ceph-mon[34674]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:10 smithi145 ceph-mon[34674]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:10 smithi145 ceph-mon[34674]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:44:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:44:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-mon[34587]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:44:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-mon[34587]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:11 smithi046 ceph-mon[30951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:44:11.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:11 smithi046 ceph-mon[30951]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:44:11.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:11.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:11 smithi046 ceph-mon[30951]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:11 smithi046 ceph-mon[30951]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.285 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:44:11.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-09-12T02:44:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:44:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:11 smithi145 ceph-mon[34674]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:44:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:11 smithi145 ceph-mon[34674]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:44:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:11 smithi145 ceph-mon[34674]: from='osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:11 smithi145 ceph-mon[34674]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:44:11.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T02:44:11.632+0000 7fd1d7248640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T02:44:11.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 02:44:11 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T02:44:11.640+0000 7fd1cf068640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:44:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 -- 172.21.15.46:0/3176025798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c107900 msgr2=0x7f668c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 --2- 172.21.15.46:0/3176025798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c107900 0x7f668c109d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6684009ca0 tx=0x7f668402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 -- 172.21.15.46:0/3176025798 shutdown_connections 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 --2- 172.21.15.46:0/3176025798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c107900 0x7f668c109d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 --2- 172.21.15.46:0/3176025798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c104fd0 0x7f668c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 --2- 172.21.15.46:0/3176025798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 0x7f668c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 -- 172.21.15.46:0/3176025798 >> 172.21.15.46:0/3176025798 conn(0x7f668c0fc2a0 msgr2=0x7f668c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 -- 172.21.15.46:0/3176025798 shutdown_connections 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.054+0000 7f669475d640 1 -- 172.21.15.46:0/3176025798 wait complete. 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 Processor -- start 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 -- start start 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 0x7f668c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c107900 0x7f668c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d7d0 con 0x7f668c104fd0 2024-09-12T02:44:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d650 con 0x7f668c1026a0 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f669475d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d950 con 0x7f668c107900 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56930/0 (socket says 172.21.15.46:56930) 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 -- 172.21.15.46:0/1758902938 learned_addr learned my addr 172.21.15.46:0/1758902938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6692cfa640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c107900 0x7f668c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 -- 172.21.15.46:0/1758902938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c107900 msgr2=0x7f668c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c107900 0x7f668c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f66924f9640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 0x7f668c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 -- 172.21.15.46:0/1758902938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 msgr2=0x7f668c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 0x7f668c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.055+0000 7f6691cf8640 1 -- 172.21.15.46:0/1758902938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c19fac0 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f6691cf8640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f667c005fd0 tx=0x7f667c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c017070 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c19fdb0 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c009c30 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c003d40 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.056+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a0260 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.057+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c0fdd90 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.058+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f667c003ee0 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.059+0000 7f66837fe640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 0x7f66780a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.059+0000 7f66924f9640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 0x7f66780a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.059+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4253+0+0 (secure 0 0 0) 0x7f667c0ce5e0 con 0x7f668c104fd0 2024-09-12T02:44:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.059+0000 7f66924f9640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 0x7f66780a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f668800a810 tx=0x7f6688008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:44:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.062+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f667c020030 con 0x7f668c104fd0 2024-09-12T02:44:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:12.249+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f668c062630 con 0x7f66780a2bb0 2024-09-12T02:44:12.251 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:12.251 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:44:12.251 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T02:44:12.251 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:12 smithi046 ceph-mon[30951]: from='osd.5 ' entity='osd.5' 2024-09-12T02:44:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:12 smithi196 ceph-mon[34587]: from='osd.5 ' entity='osd.5' 2024-09-12T02:44:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:44:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T02:44:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:12.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:12 smithi145 ceph-mon[34674]: from='osd.5 ' entity='osd.5' 2024-09-12T02:44:13.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi196 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] boot 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:13.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi196 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] boot 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:13.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi196 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: osd.5 [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] boot 2024-09-12T02:44:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:13.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:14.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:14 smithi046 ceph-mon[30951]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:44:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:14 smithi196 ceph-mon[34587]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:44:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:14 smithi145 ceph-mon[34674]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:44:15.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]: dispatch 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]': finished 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:44:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:44:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:44:15.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]: dispatch 2024-09-12T02:44:15.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]': finished 2024-09-12T02:44:15.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:44:15.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]: dispatch 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3302677419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "499ded47-5f1d-433f-8a32-c75b6b64106a"}]': finished 2024-09-12T02:44:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:44:15.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:16 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3763271626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:16 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3763271626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:16 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3763271626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:17.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:17 smithi196 ceph-mon[34587]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:17.511 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:17 smithi145 ceph-mon[34674]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:17 smithi046 ceph-mon[30951]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:19 smithi046 ceph-mon[30951]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:19.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:19 smithi196 ceph-mon[34587]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:19 smithi145 ceph-mon[34674]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:21.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:21 smithi196 ceph-mon[34587]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:44:21.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:21 smithi046 ceph-mon[30951]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:44:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:21 smithi145 ceph-mon[34674]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:44:23.482 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:23 smithi046 ceph-mon[30951]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:44:23.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:23 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:44:23.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:23 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:23.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:23 smithi046 ceph-mon[30951]: Deploying daemon osd.6 on smithi046 2024-09-12T02:44:23.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:23 smithi145 ceph-mon[34674]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:44:23.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:23 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:44:23.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:23 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:23.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:23 smithi145 ceph-mon[34674]: Deploying daemon osd.6 on smithi046 2024-09-12T02:44:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:23 smithi196 ceph-mon[34587]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:44:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:23 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:44:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:23 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:23.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:23 smithi196 ceph-mon[34587]: Deploying daemon osd.6 on smithi046 2024-09-12T02:44:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:25 smithi046 ceph-mon[30951]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T02:44:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:25 smithi145 ceph-mon[34674]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T02:44:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:25 smithi196 ceph-mon[34587]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T02:44:27.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:27 smithi046 ceph-mon[30951]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:44:27.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:27 smithi145 ceph-mon[34674]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:44:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:27 smithi196 ceph-mon[34587]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:44:29.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:29 smithi046 ceph-mon[30951]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:29 smithi145 ceph-mon[34674]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:29 smithi196 ceph-mon[34587]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:30.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:30.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:30.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:30.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:31.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:31 smithi046 ceph-mon[30951]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:31 smithi145 ceph-mon[34674]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:31 smithi196 ceph-mon[34587]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:44:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.278+0000 7f66837fe640 1 -- 172.21.15.46:0/1758902938 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f668c062630 con 0x7f66780a2bb0 2024-09-12T02:44:33.280 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.280+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 msgr2=0x7f66780a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.280+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 0x7f66780a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f668800a810 tx=0x7f6688008040 comp rx=0 tx=0).stop 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.280+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 msgr2=0x7f668c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.280+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f667c005fd0 tx=0x7f667c00dde0 comp rx=0 tx=0).stop 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 shutdown_connections 2024-09-12T02:44:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f66780a2bb0 0x7f66780a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f668c107900 0x7f668c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f668c104fd0 0x7f668c1983d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 --2- 172.21.15.46:0/1758902938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f668c1026a0 0x7f668c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 >> 172.21.15.46:0/1758902938 conn(0x7f668c0fc2a0 msgr2=0x7f668c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 shutdown_connections 2024-09-12T02:44:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:44:33.281+0000 7f669475d640 1 -- 172.21.15.46:0/1758902938 wait complete. 2024-09-12T02:44:33.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:33.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:33.954 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T02:44:33.957 INFO:tasks.cephadm:Deploying osd.7 on smithi145 with /dev/vg_nvme/lv_2... 2024-09-12T02:44:33.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:44:34.222 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:44:35.419 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:35 smithi046 ceph-mon[30951]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:35.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:35 smithi145 ceph-mon[34674]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:35.713 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 02:44:35 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T02:44:35.416+0000 7f5cd3258640 -1 osd.6 0 log_to_monitors true 2024-09-12T02:44:35.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:35 smithi196 ceph-mon[34587]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:36 smithi046 ceph-mon[30951]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:44:36.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:36 smithi145 ceph-mon[34674]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:44:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:36 smithi196 ceph-mon[34587]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:44:36.984 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:44:37.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-09-12T02:44:37.263 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:44:37.551 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:37.551 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:44:37.551 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:44:37.551 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:44:37.551 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi046 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 5697M 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:37.552 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi046 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 5697M 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.783 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T02:44:37.301+0000 7f5ccf1db640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T02:44:37.783 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T02:44:37.310+0000 7f5ccb003640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi046 2024-09-12T02:44:37.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 5697M 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:37.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:37.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 -- 172.21.15.145:0/4140160767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18741037c0 msgr2=0x7f187410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:37.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 --2- 172.21.15.145:0/4140160767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18741037c0 0x7f187410a050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f186c009ca0 tx=0x7f186c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 -- 172.21.15.145:0/4140160767 shutdown_connections 2024-09-12T02:44:37.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 --2- 172.21.15.145:0/4140160767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18741037c0 0x7f187410a050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 --2- 172.21.15.145:0/4140160767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1874102e00 0x7f1874103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 --2- 172.21.15.145:0/4140160767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874101c00 0x7f1874102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 -- 172.21.15.145:0/4140160767 >> 172.21.15.145:0/4140160767 conn(0x7f18740fd390 msgr2=0x7f18740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 -- 172.21.15.145:0/4140160767 shutdown_connections 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.790+0000 7f187a3c7640 1 -- 172.21.15.145:0/4140160767 wait complete. 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 Processor -- start 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 -- start start 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874102e00 0x7f18740745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741037c0 0x7f1874074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d780 con 0x7f18741037c0 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d600 con 0x7f1874101c00 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f187a3c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d900 con 0x7f1874102e00 2024-09-12T02:44:37.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f1873fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f1873fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56920/0 (socket says 172.21.15.145:56920) 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.791+0000 7f1873fff640 1 -- 172.21.15.145:0/2573943278 learned_addr learned my addr 172.21.15.145:0/2573943278 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1878964640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741037c0 0x7f1874074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f18737fe640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874102e00 0x7f18740745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 -- 172.21.15.145:0/2573943278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874102e00 msgr2=0x7f18740745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874102e00 0x7f18740745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 -- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741037c0 msgr2=0x7f1874074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741037c0 0x7f1874074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 -- 172.21.15.145:0/2573943278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740753e0 con 0x7f1874101c00 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f1873fff640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f186400d950 tx=0x7f186400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:44:37.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864017070 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1864009c30 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a1ee0 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186400f620 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.792+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a2280 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.793+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1874108f30 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.794+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f186401a070 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.795+0000 7f18717fa640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 0x7f185c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.795+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4685+0+0 (secure 0 0 0) 0x7f18640cdcf0 con 0x7f1874101c00 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.795+0000 7f18737fe640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 0x7f185c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.796+0000 7f18737fe640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 0x7f185c0a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1868005e10 tx=0x7f1868005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:44:37.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.798+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1864009de0 con 0x7f1874101c00 2024-09-12T02:44:37.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:37.977+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1874062630 con 0x7f185c0a2b80 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:38.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:44:38.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:38.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:44:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:44:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:38.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:44:38.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:44:38.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:44:38.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] boot 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:44:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:44:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:44:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] boot 2024-09-12T02:44:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:44:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: osd.6 [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] boot 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:44:39.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:44:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:40 smithi145 ceph-mon[34674]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:44:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:40 smithi196 ceph-mon[34587]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:44:40.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:40 smithi046 ceph-mon[30951]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/1013830787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]': finished 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:44:41.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:41 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/292220442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/1013830787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]': finished 2024-09-12T02:44:41.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:44:41.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:44:41.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:41 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/292220442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/1013830787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]: dispatch 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7de3c410-22b1-47ea-94ef-93b756a84fc8"}]': finished 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:44:41.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:41 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/292220442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:44:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:43 smithi145 ceph-mon[34674]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:43 smithi196 ceph-mon[34587]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:43.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:43 smithi046 ceph-mon[30951]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:45 smithi145 ceph-mon[34674]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:45.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:45 smithi196 ceph-mon[34587]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:45.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:45 smithi046 ceph-mon[30951]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:47.550 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:47 smithi145 ceph-mon[34674]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:47.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:47 smithi196 ceph-mon[34587]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:47.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:47 smithi046 ceph-mon[30951]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:49 smithi145 ceph-mon[34674]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:49 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:44:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:49 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:49 smithi145 ceph-mon[34674]: Deploying daemon osd.7 on smithi145 2024-09-12T02:44:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:49 smithi196 ceph-mon[34587]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:49 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:44:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:49 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:49.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:49 smithi196 ceph-mon[34587]: Deploying daemon osd.7 on smithi145 2024-09-12T02:44:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:49 smithi046 ceph-mon[30951]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:49 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:44:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:49 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:49 smithi046 ceph-mon[30951]: Deploying daemon osd.7 on smithi145 2024-09-12T02:44:51.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:51 smithi145 ceph-mon[34674]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:51.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:51 smithi196 ceph-mon[34587]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:51.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:51 smithi046 ceph-mon[30951]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:53.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:53 smithi145 ceph-mon[34674]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:53.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:53 smithi196 ceph-mon[34587]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:53.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:53 smithi046 ceph-mon[30951]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:55 smithi145 ceph-mon[34674]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:55.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:55 smithi196 ceph-mon[34587]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:55.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:55.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:55.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:55 smithi046 ceph-mon[30951]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:55.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:44:55.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:55.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:57.450 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:57 smithi145 ceph-mon[34674]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:57 smithi196 ceph-mon[34587]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:57.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:57 smithi046 ceph-mon[30951]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:44:59.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.113+0000 7f18717fa640 1 -- 172.21.15.145:0/2573943278 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1874062630 con 0x7f185c0a2b80 2024-09-12T02:44:59.114 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 7 on host 'smithi145' 2024-09-12T02:44:59.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 msgr2=0x7f185c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 0x7f185c0a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1868005e10 tx=0x7f1868005da0 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 msgr2=0x7f1874075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f186400d950 tx=0x7f186400de10 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 shutdown_connections 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741037c0 0x7f1874074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f185c0a2b80 0x7f185c0a5040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874102e00 0x7f18740745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 --2- 172.21.15.145:0/2573943278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874101c00 0x7f1874075f90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 >> 172.21.15.145:0/2573943278 conn(0x7f18740fd390 msgr2=0x7f1874106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:44:59.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 shutdown_connections 2024-09-12T02:44:59.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:44:59.115+0000 7f187a3c7640 1 -- 172.21.15.145:0/2573943278 wait complete. 2024-09-12T02:44:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:59.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:44:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:59.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:59.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:44:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:44:59.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:44:59.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:44:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:44:59.762 DEBUG:teuthology.orchestra.run.smithi145:osd.7> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T02:44:59.764 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-09-12T02:44:59.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:45:00.019 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:45:00.142 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:00 smithi196 ceph-mon[34587]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:00.142 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:00 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:00.142 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:00 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:00.142 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:00 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:00.322 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:00 smithi145 ceph-mon[34674]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:00.322 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:00 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:00.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:00 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:00.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:00 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:00 smithi046 ceph-mon[30951]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:00 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:00 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:00 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:01.036 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 02:45:00 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T02:45:00.773+0000 7fa71049e640 -1 osd.7 0 log_to_monitors true 2024-09-12T02:45:01.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:01 smithi145 ceph-mon[34674]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:01.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:01 smithi145 ceph-mon[34674]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:01.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:01 smithi196 ceph-mon[34587]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:01.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:01 smithi196 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:01.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:01 smithi046 ceph-mon[30951]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:01.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:01 smithi046 ceph-mon[30951]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.343 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-mon[34674]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.409 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:02 smithi196 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: from='osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:02 smithi046 ceph-mon[30951]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:45:02.610 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T02:45:02.429+0000 7fa70c421640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T02:45:02.610 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 02:45:02 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T02:45:02.438+0000 7fa704241640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:45:02.830 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:45:02.858 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-09-12T02:45:03.108 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:45:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:45:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='osd.7 ' entity='osd.7' 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:03 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:45:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T02:45:03.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='osd.7 ' entity='osd.7' 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:03.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:03.362 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:03 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='osd.7 ' entity='osd.7' 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:03.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:03 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:03.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- 172.21.15.196:0/573047612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec1037c0 msgr2=0x7f71ec10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:03.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 --2- 172.21.15.196:0/573047612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec1037c0 0x7f71ec10a050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009ca0 tx=0x7f71e402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- 172.21.15.196:0/573047612 shutdown_connections 2024-09-12T02:45:03.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 --2- 172.21.15.196:0/573047612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec1037c0 0x7f71ec10a050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 --2- 172.21.15.196:0/573047612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ec102e00 0x7f71ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 --2- 172.21.15.196:0/573047612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- 172.21.15.196:0/573047612 >> 172.21.15.196:0/573047612 conn(0x7f71ec0fd390 msgr2=0x7f71ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- 172.21.15.196:0/573047612 shutdown_connections 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- 172.21.15.196:0/573047612 wait complete. 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 Processor -- start 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.654+0000 7f71f316c640 1 -- start start 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ec1037c0 0x7f71ec19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d730 con 0x7f71ec1037c0 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d5b0 con 0x7f71ec101c00 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d8b0 con 0x7f71ec102e00 2024-09-12T02:45:03.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:57958/0 (socket says 172.21.15.196:57958) 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 -- 172.21.15.196:0/1597609026 learned_addr learned my addr 172.21.15.196:0/1597609026 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f0f08640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 -- 172.21.15.196:0/1597609026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 msgr2=0x7f71ec197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 -- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ec1037c0 msgr2=0x7f71ec19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ec1037c0 0x7f71ec19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 -- 172.21.15.196:0/1597609026 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec19f9c0 con 0x7f71ec102e00 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f0f08640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71ebfff640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c920 tx=0x7f71e000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0007c70 con 0x7f71ec102e00 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec19fcb0 con 0x7f71ec102e00 2024-09-12T02:45:03.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e0007e10 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.655+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0018820 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.656+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec1a01a0 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.657+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec108fa0 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.658+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71e00189c0 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.658+0000 7f71e9ffb640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 0x7f71d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.658+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5430+0+0 (secure 0 0 0) 0x7f71e00ce720 con 0x7f71ec102e00 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.658+0000 7f71f0f08640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 0x7f71d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.659+0000 7f71f0f08640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 0x7f71d40a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005fd0 tx=0x7f71dc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:03.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.663+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71e0014030 con 0x7f71ec102e00 2024-09-12T02:45:03.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:03.838+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f71ec062630 con 0x7f71d40a2b80 2024-09-12T02:45:04.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi145 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] boot 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:04.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:04 smithi196 ceph-mon[34587]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:45:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:45:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:45:04.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi145 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] boot 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:04.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:04 smithi145 ceph-mon[34674]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi145 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: osd.7 [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] boot 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:45:04.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:45:04.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:04.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:04.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:04.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:04 smithi046 ceph-mon[30951]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:45:05.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:05 smithi196 ceph-mon[34587]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:05.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:05 smithi196 ceph-mon[34587]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:45:05.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:05 smithi145 ceph-mon[34674]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:05.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:05 smithi145 ceph-mon[34674]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:45:05.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:05 smithi046 ceph-mon[30951]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:05 smithi046 ceph-mon[30951]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:45:06.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:06 smithi196 ceph-mon[34587]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:06.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:06 smithi145 ceph-mon[34674]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:06 smithi046 ceph-mon[30951]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:07.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T02:45:07.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/4178713694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]': finished 2024-09-12T02:45:07.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:45:07.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:07.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:07 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/727214072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:07.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T02:45:07.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/4178713694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]': finished 2024-09-12T02:45:07.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:45:07.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:07.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:07 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/727214072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/4178713694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]: dispatch 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040d8985-a30f-4bdd-a469-0be6e29b98a8"}]': finished 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:07.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:07 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/727214072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:08.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:08 smithi145 ceph-mon[34674]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:08.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:08 smithi046 ceph-mon[30951]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:08.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:08 smithi196 ceph-mon[34587]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:10 smithi145 ceph-mon[34674]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:10.382 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:10 smithi196 ceph-mon[34587]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:10 smithi046 ceph-mon[30951]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:12.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:12 smithi196 ceph-mon[34587]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:12.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:12 smithi145 ceph-mon[34674]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:12.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:12 smithi046 ceph-mon[30951]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:14.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:14 smithi196 ceph-mon[34587]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:14.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:14 smithi145 ceph-mon[34674]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:14 smithi046 ceph-mon[30951]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:45:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:15 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:45:15.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:15 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:15.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:45:15.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:15 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:16 smithi145 ceph-mon[34674]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:16 smithi145 ceph-mon[34674]: Deploying daemon osd.8 on smithi196 2024-09-12T02:45:16.396 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:16 smithi196 ceph-mon[34587]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:16.396 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:16 smithi196 ceph-mon[34587]: Deploying daemon osd.8 on smithi196 2024-09-12T02:45:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:16 smithi046 ceph-mon[30951]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:16 smithi046 ceph-mon[30951]: Deploying daemon osd.8 on smithi196 2024-09-12T02:45:18.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:18 smithi145 ceph-mon[34674]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:18.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:18 smithi196 ceph-mon[34587]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:18 smithi046 ceph-mon[30951]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:20.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:20 smithi196 ceph-mon[34587]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:20.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:20 smithi145 ceph-mon[34674]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:20 smithi046 ceph-mon[30951]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:21.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:21.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:21.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:21.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:21.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:21.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:22.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:22 smithi145 ceph-mon[34674]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:22.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:22 smithi196 ceph-mon[34587]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:22 smithi046 ceph-mon[30951]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:24.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:24.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:24 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:24 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:24 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:24.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.705+0000 7f71e9ffb640 1 -- 172.21.15.196:0/1597609026 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71ec062630 con 0x7f71d40a2b80 2024-09-12T02:45:24.706 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 msgr2=0x7f71d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 0x7f71d40a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005fd0 tx=0x7f71dc005850 comp rx=0 tx=0).stop 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 msgr2=0x7f71ec1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c920 tx=0x7f71e000cde0 comp rx=0 tx=0).stop 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 shutdown_connections 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71ec1037c0 0x7f71ec19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:24.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f71d40a2b80 0x7f71d40a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:24.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71ec102e00 0x7f71ec1982e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:24.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 --2- 172.21.15.196:0/1597609026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71ec101c00 0x7f71ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:24.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 >> 172.21.15.196:0/1597609026 conn(0x7f71ec0fd390 msgr2=0x7f71ec106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:45:24.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 shutdown_connections 2024-09-12T02:45:24.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:45:24.708+0000 7f71f316c640 1 -- 172.21.15.196:0/1597609026 wait complete. 2024-09-12T02:45:25.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:25 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:25.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:25 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:25 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.395 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:25.395 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.395 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.430 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T02:45:25.433 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-09-12T02:45:25.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:45:25.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:25.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:45:26.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:26 smithi145 ceph-mon[34674]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:26.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:26 smithi196 ceph-mon[34587]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:26 smithi046 ceph-mon[30951]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:26.714 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 02:45:26 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T02:45:26.372+0000 7fd706f4b640 -1 osd.8 0 log_to_monitors true 2024-09-12T02:45:27.250 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:27 smithi046 ceph-mon[30951]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:27.250 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:27 smithi046 ceph-mon[30951]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:27.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:27 smithi145 ceph-mon[34674]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:27.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:27 smithi145 ceph-mon[34674]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:27.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:27 smithi196 ceph-mon[34587]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:27.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:27 smithi196 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:28 smithi046 ceph-mon[30951]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:28.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: from='osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:28 smithi145 ceph-mon[34674]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:45:28.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T02:45:28.510+0000 7fd702ece640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T02:45:28.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 02:45:28 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T02:45:28.520+0000 7fd6facee640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:45:28.821 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:45:28.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-09-12T02:45:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:29 smithi046 ceph-mon[30951]: from='osd.8 ' entity='osd.8' 2024-09-12T02:45:29.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:45:29.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:29 smithi196 ceph-mon[34587]: from='osd.8 ' entity='osd.8' 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:29.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:29 smithi145 ceph-mon[34674]: from='osd.8 ' entity='osd.8' 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 -- 172.21.15.46:0/3585469624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8102690 msgr2=0x7f1bd8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 --2- 172.21.15.46:0/3585469624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8102690 0x7f1bd8108f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009ca0 tx=0x7f1bd002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 -- 172.21.15.46:0/3585469624 shutdown_connections 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 --2- 172.21.15.46:0/3585469624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8102690 0x7f1bd8108f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 --2- 172.21.15.46:0/3585469624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8101cd0 0x7f1bd8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 --2- 172.21.15.46:0/3585469624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8100ad0 0x7f1bd8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 -- 172.21.15.46:0/3585469624 >> 172.21.15.46:0/3585469624 conn(0x7f1bd80fc280 msgr2=0x7f1bd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 -- 172.21.15.46:0/3585469624 shutdown_connections 2024-09-12T02:45:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.651+0000 7f1bddd28640 1 -- 172.21.15.46:0/3585469624 wait complete. 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 Processor -- start 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 -- start start 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8100ad0 0x7f1bd8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 0x7f1bd819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810c5e0 con 0x7f1bd8101cd0 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810c460 con 0x7f1bd8100ad0 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bddd28640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810c760 con 0x7f1bd8102690 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bcffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bcffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41010/0 (socket says 172.21.15.46:41010) 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bcffff640 1 -- 172.21.15.46:0/4047040599 learned_addr learned my addr 172.21.15.46:0/4047040599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bd7fff640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 0x7f1bd819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.652+0000 7f1bd77fe640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8100ad0 0x7f1bd8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 -- 172.21.15.46:0/4047040599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 msgr2=0x7f1bd819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 0x7f1bd819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 -- 172.21.15.46:0/4047040599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8100ad0 msgr2=0x7f1bd8196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8100ad0 0x7f1bd8196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 -- 172.21.15.46:0/4047040599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd819e7b0 con 0x7f1bd8101cd0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bd7fff640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 0x7f1bd819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bcffff640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c920 tx=0x7f1bc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0007c70 con 0x7f1bd8101cd0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bc0007e10 con 0x7f1bd8101cd0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd819eaa0 con 0x7f1bd8101cd0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0018820 con 0x7f1bd8101cd0 2024-09-12T02:45:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.653+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd819f030 con 0x7f1bd8101cd0 2024-09-12T02:45:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.654+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd8107e00 con 0x7f1bd8101cd0 2024-09-12T02:45:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.655+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1bc00189c0 con 0x7f1bd8101cd0 2024-09-12T02:45:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.656+0000 7f1bd57fa640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 0x7f1bc40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.656+0000 7f1bd77fe640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 0x7f1bc40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.656+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7f1bc00ce8b0 con 0x7f1bd8101cd0 2024-09-12T02:45:29.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.657+0000 7f1bd77fe640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 0x7f1bc40a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80058c0 tx=0x7f1bc8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.661+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1bc0014030 con 0x7f1bd8101cd0 2024-09-12T02:45:29.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:29.845+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1bd8062630 con 0x7f1bc40a2c30 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi196 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-12T02:45:30.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] boot 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:30.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:30 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:30.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi196 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] boot 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:30.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:30 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:45:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:45:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi196 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: osd.8 [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] boot 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:30.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:30 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:31.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:31 smithi046 ceph-mon[30951]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:31.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:31 smithi046 ceph-mon[30951]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:45:31.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:31 smithi196 ceph-mon[34587]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:31.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:31 smithi196 ceph-mon[34587]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:45:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:31 smithi145 ceph-mon[34674]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:31 smithi145 ceph-mon[34674]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:45:32.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:32 smithi196 ceph-mon[34587]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:32.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:32 smithi046 ceph-mon[30951]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:32 smithi145 ceph-mon[34674]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:33.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T02:45:33.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]: dispatch 2024-09-12T02:45:33.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]': finished 2024-09-12T02:45:33.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:45:33.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:33.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:33 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3213786192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]: dispatch 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]': finished 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:33.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3213786192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T02:45:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]: dispatch 2024-09-12T02:45:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2348949117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5445403e-05c9-4020-b773-336395f26213"}]': finished 2024-09-12T02:45:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:45:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:33.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:33 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3213786192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:34 smithi196 ceph-mon[34587]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:34.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:34 smithi046 ceph-mon[30951]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:34 smithi145 ceph-mon[34674]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:36 smithi196 ceph-mon[34587]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:45:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:36 smithi046 ceph-mon[30951]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:45:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:36 smithi145 ceph-mon[34674]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T02:45:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:38 smithi196 ceph-mon[34587]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:45:38.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:38 smithi046 ceph-mon[30951]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:45:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:38 smithi145 ceph-mon[34674]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:45:40.362 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:40 smithi046 ceph-mon[30951]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:45:40.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:40 smithi196 ceph-mon[34587]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:45:40.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:40 smithi145 ceph-mon[34674]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:45:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:42 smithi196 ceph-mon[34587]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:45:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:45:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:42 smithi046 ceph-mon[30951]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:45:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:45:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:42 smithi145 ceph-mon[34674]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:45:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:45:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:43.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:43 smithi046 ceph-mon[30951]: Deploying daemon osd.9 on smithi046 2024-09-12T02:45:43.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:43 smithi196 ceph-mon[34587]: Deploying daemon osd.9 on smithi046 2024-09-12T02:45:43.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:43 smithi145 ceph-mon[34674]: Deploying daemon osd.9 on smithi046 2024-09-12T02:45:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:44 smithi196 ceph-mon[34587]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:45:44.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:44 smithi046 ceph-mon[30951]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:45:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:44 smithi145 ceph-mon[34674]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:45:46.263 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:46 smithi046 ceph-mon[30951]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:45:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:46 smithi196 ceph-mon[34587]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:45:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:46 smithi145 ceph-mon[34674]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:45:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:48 smithi145 ceph-mon[34674]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:48.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:48.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:48.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:48.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:48 smithi046 ceph-mon[30951]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:48.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:48.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:48.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:48 smithi196 ceph-mon[34587]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:48.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:50 smithi196 ceph-mon[34587]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:50.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:50 smithi046 ceph-mon[30951]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:50 smithi145 ceph-mon[34674]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.197+0000 7f1bd57fa640 1 -- 172.21.15.46:0/4047040599 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1bd8062630 con 0x7f1bc40a2c30 2024-09-12T02:45:51.200 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-09-12T02:45:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 msgr2=0x7f1bc40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 0x7f1bc40a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80058c0 tx=0x7f1bc8005850 comp rx=0 tx=0).stop 2024-09-12T02:45:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 msgr2=0x7f1bd81970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c920 tx=0x7f1bc000cde0 comp rx=0 tx=0).stop 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 shutdown_connections 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1bc40a2c30 0x7f1bc40a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd8102690 0x7f1bd819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd8101cd0 0x7f1bd81970c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 --2- 172.21.15.46:0/4047040599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bd8100ad0 0x7f1bd8196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 >> 172.21.15.46:0/4047040599 conn(0x7f1bd80fc280 msgr2=0x7f1bd8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 shutdown_connections 2024-09-12T02:45:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:45:51.200+0000 7f1bddd28640 1 -- 172.21.15.46:0/4047040599 wait complete. 2024-09-12T02:45:51.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:51 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.848 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T02:45:51.851 INFO:tasks.cephadm:Deploying osd.10 on smithi145 with /dev/vg_nvme/lv_1... 2024-09-12T02:45:51.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:51 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:51.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:51.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:45:51.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:51.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:51 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:52.105 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:45:53.533 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 02:45:53 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T02:45:53.476+0000 7f02c000d640 -1 osd.9 0 log_to_monitors true 2024-09-12T02:45:53.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:53 smithi145 ceph-mon[34674]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:53.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:53 smithi145 ceph-mon[34674]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:45:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:53 smithi196 ceph-mon[34587]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:53 smithi196 ceph-mon[34587]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:45:54.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:53 smithi046 ceph-mon[30951]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:54.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:53 smithi046 ceph-mon[30951]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:45:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-mon[30951]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:45:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-mon[30951]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:45:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-mon[30951]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:45:54.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:54.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-mon[30951]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:45:54.783 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T02:45:54.563+0000 7f02bbf90640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T02:45:54.783 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 02:45:54 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T02:45:54.576+0000 7f02b7db8640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:45:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:54 smithi145 ceph-mon[34674]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:45:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:54 smithi145 ceph-mon[34674]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:45:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:54 smithi145 ceph-mon[34674]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:45:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:54 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:54 smithi145 ceph-mon[34674]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:45:54.868 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-12T02:45:54.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-09-12T02:45:54.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:54 smithi196 ceph-mon[34587]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:45:54.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:54 smithi196 ceph-mon[34587]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:45:54.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:54 smithi196 ceph-mon[34587]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-12T02:45:54.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:54 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:54.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:54 smithi196 ceph-mon[34587]: from='osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-12T02:45:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 -- 172.21.15.145:0/3667313428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c100b00 msgr2=0x7f0e1c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 --2- 172.21.15.145:0/3667313428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c100b00 0x7f0e1c100f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02a410 comp rx=0 tx=0).stop 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 -- 172.21.15.145:0/3667313428 shutdown_connections 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 --2- 172.21.15.145:0/3667313428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c1026c0 0x7f0e1c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 --2- 172.21.15.145:0/3667313428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c101d00 0x7f0e1c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:55.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 --2- 172.21.15.145:0/3667313428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c100b00 0x7f0e1c100f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 -- 172.21.15.145:0/3667313428 >> 172.21.15.145:0/3667313428 conn(0x7f0e1c0fc290 msgr2=0x7f0e1c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 -- 172.21.15.145:0/3667313428 shutdown_connections 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.675+0000 7f0e20f47640 1 -- 172.21.15.145:0/3667313428 wait complete. 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 Processor -- start 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 -- start start 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 0x7f0e1c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c1026c0 0x7f0e1c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c10c630 con 0x7f0e1c100b00 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c10c4b0 con 0x7f0e1c101d00 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e20f47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c10c7b0 con 0x7f0e1c1026c0 2024-09-12T02:45:55.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e19d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e19d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50588/0 (socket says 172.21.15.145:50588) 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.676+0000 7f0e19d9b640 1 -- 172.21.15.145:0/139317005 learned_addr learned my addr 172.21.15.145:0/139317005 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e1ad9d640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c1026c0 0x7f0e1c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 -- 172.21.15.145:0/139317005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c1026c0 msgr2=0x7f0e1c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e1a59c640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 0x7f0e1c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c1026c0 0x7f0e1c19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 -- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 msgr2=0x7f0e1c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 0x7f0e1c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 -- 172.21.15.145:0/139317005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c19fa10 con 0x7f0e1c101d00 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e1a59c640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 0x7f0e1c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e19d9b640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000cd80 con 0x7f0e1c101d00 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c19fd00 con 0x7f0e1c101d00 2024-09-12T02:45:55.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e100054d0 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10010490 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.677+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1a01e0 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.678+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c108ae0 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.679+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e10005000 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.680+0000 7f0e0b7fe640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 0x7f0e040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.680+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7f0e100ceed0 con 0x7f0e1c101d00 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.680+0000 7f0e1a59c640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 0x7f0e040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.681+0000 7f0e1a59c640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 0x7f0e040a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:45:55.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.683+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e100541b0 con 0x7f0e1c101d00 2024-09-12T02:45:55.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:45:55.857+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0e1c062630 con 0x7f0e040a2bb0 2024-09-12T02:45:55.858 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:55.858 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T02:45:55.858 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:55.858 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi046 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:55 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:55.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi046 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-12T02:45:55.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:55.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:55.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:55.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:55 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T02:45:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi046 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 4273M 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:45:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:45:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:55 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:45:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:45:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] boot 2024-09-12T02:45:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:45:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:56.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:45:56.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:45:56.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] boot 2024-09-12T02:45:56.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:45:56.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:56.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:56.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:56.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:56.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:57.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:45:57.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:45:57.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: osd.9 [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] boot 2024-09-12T02:45:57.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:45:57.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:45:57.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:45:57.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:45:57.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:45:57.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:45:57.750 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:57 smithi145 ceph-mon[34674]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:57.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:57 smithi145 ceph-mon[34674]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T02:45:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:57 smithi196 ceph-mon[34587]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:57 smithi196 ceph-mon[34587]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T02:45:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:57 smithi046 ceph-mon[30951]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:45:58.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:57 smithi046 ceph-mon[30951]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T02:45:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:58 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/2360048875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:58 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:58 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]': finished 2024-09-12T02:45:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:58 smithi145 ceph-mon[34674]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:45:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:58 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:45:58.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:58 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/2360048875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:58.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:58 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:58.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:58 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]': finished 2024-09-12T02:45:58.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:58 smithi196 ceph-mon[34587]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:45:58.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:58 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:45:59.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:58 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/2360048875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:59.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:58 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]: dispatch 2024-09-12T02:45:59.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:58 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5fbcc26-7505-434d-b947-79310be95234"}]': finished 2024-09-12T02:45:59.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:58 smithi046 ceph-mon[30951]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:45:59.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:58 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:45:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:59 smithi145 ceph-mon[34674]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:45:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:45:59 smithi145 ceph-mon[34674]: from='client.? 172.21.15.145:0/2494193054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:45:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:59 smithi196 ceph-mon[34587]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:45:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:45:59 smithi196 ceph-mon[34587]: from='client.? 172.21.15.145:0/2494193054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:46:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:59 smithi046 ceph-mon[30951]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:45:59 smithi046 ceph-mon[30951]: from='client.? 172.21.15.145:0/2494193054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:46:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:01 smithi145 ceph-mon[34674]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:01.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:01 smithi196 ceph-mon[34587]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:02.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:01 smithi046 ceph-mon[30951]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:03 smithi145 ceph-mon[34674]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:03 smithi196 ceph-mon[34587]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:04.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:03 smithi046 ceph-mon[30951]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:05.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:05 smithi145 ceph-mon[34674]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:05 smithi196 ceph-mon[34587]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:06.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:05 smithi046 ceph-mon[30951]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:07 smithi145 ceph-mon[34674]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:46:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:07 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:07 smithi196 ceph-mon[34587]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:46:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:07 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:07 smithi046 ceph-mon[30951]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:46:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:07 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:08.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:08 smithi145 ceph-mon[34674]: Deploying daemon osd.10 on smithi145 2024-09-12T02:46:08.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:08 smithi046 ceph-mon[30951]: Deploying daemon osd.10 on smithi145 2024-09-12T02:46:08.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:08 smithi196 ceph-mon[34587]: Deploying daemon osd.10 on smithi145 2024-09-12T02:46:09.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:09 smithi145 ceph-mon[34674]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:09 smithi196 ceph-mon[34587]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:10.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:09 smithi046 ceph-mon[30951]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:11.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:11 smithi196 ceph-mon[34587]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:12.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:11 smithi046 ceph-mon[30951]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:12.105 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:11 smithi145 ceph-mon[34674]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:13.882 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:13 smithi145 ceph-mon[34674]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:13.883 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:13.883 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:13.883 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:13 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:13 smithi196 ceph-mon[34587]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:13.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:13.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:13 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:14.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:13 smithi046 ceph-mon[30951]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:14.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:14.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:14.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:13 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:15.899 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:15 smithi145 ceph-mon[34674]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:15 smithi196 ceph-mon[34587]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:15 smithi046 ceph-mon[30951]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:17.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.184+0000 7f0e0b7fe640 1 -- 172.21.15.145:0/139317005 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0e1c062630 con 0x7f0e040a2bb0 2024-09-12T02:46:17.185 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 10 on host 'smithi145' 2024-09-12T02:46:17.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 msgr2=0x7f0e040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 0x7f0e040a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 msgr2=0x7f0e1c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000edf0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 shutdown_connections 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e040a2bb0 0x7f0e040a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c1026c0 0x7f0e1c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e1c101d00 0x7f0e1c198320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 --2- 172.21.15.145:0/139317005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c100b00 0x7f0e1c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 >> 172.21.15.145:0/139317005 conn(0x7f0e1c0fc290 msgr2=0x7f0e1c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 shutdown_connections 2024-09-12T02:46:17.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-12T02:46:17.186+0000 7f0e20f47640 1 -- 172.21.15.145:0/139317005 wait complete. 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.568 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:17.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:18.017 DEBUG:teuthology.orchestra.run.smithi145:osd.10> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T02:46:18.019 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-09-12T02:46:18.020 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:46:18.276 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:46:19.423 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:19 smithi145 ceph-mon[34674]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:19.423 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:19 smithi145 ceph-mon[34674]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:19.423 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:19 smithi145 ceph-mon[34674]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:19.423 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 02:46:19 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T02:46:19.229+0000 7f5c7324c640 -1 osd.10 0 log_to_monitors true 2024-09-12T02:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:19 smithi046 ceph-mon[30951]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:19 smithi046 ceph-mon[30951]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:19 smithi046 ceph-mon[30951]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:19.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:19 smithi196 ceph-mon[34587]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:19.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:19 smithi196 ceph-mon[34587]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:19.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:19 smithi196 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:46:20.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T02:46:20.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T02:46:20.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T02:46:20.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:20.861 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T02:46:20.676+0000 7f5c6f1cf640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T02:46:20.861 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 02:46:20 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T02:46:20.688+0000 7f5c66fef640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:46:21.058 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-12T02:46:21.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-09-12T02:46:21.332 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='osd.10 ' entity='osd.10' 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi145 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] boot 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:46:21.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:21 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='osd.10 ' entity='osd.10' 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi145 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] boot 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:46:21.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:21 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='osd.10 ' entity='osd.10' 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi145 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: osd.10 [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] boot 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:46:21.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:21 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:46:21.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.819+0000 7fd929a8a640 1 -- 172.21.15.196:0/666634224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924101c00 msgr2=0x7fd924102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:21.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.819+0000 7fd929a8a640 1 --2- 172.21.15.196:0/666634224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924101c00 0x7fd924102000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd914009900 tx=0x7fd91402a410 comp rx=0 tx=0).stop 2024-09-12T02:46:21.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- 172.21.15.196:0/666634224 shutdown_connections 2024-09-12T02:46:21.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- 172.21.15.196:0/666634224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 0x7fd92410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- 172.21.15.196:0/666634224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924102e00 0x7fd924103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- 172.21.15.196:0/666634224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924101c00 0x7fd924102000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- 172.21.15.196:0/666634224 >> 172.21.15.196:0/666634224 conn(0x7fd9240fd390 msgr2=0x7fd9240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- 172.21.15.196:0/666634224 shutdown_connections 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- 172.21.15.196:0/666634224 wait complete. 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 Processor -- start 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- start start 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 0x7fd924197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 0x7fd92419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd92410d740 con 0x7fd924101c00 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd92410d5c0 con 0x7fd9241037c0 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.820+0000 7fd929a8a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd92410d8c0 con 0x7fd924102e00 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:36968/0 (socket says 172.21.15.196:36968) 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 -- 172.21.15.196:0/119620715 learned_addr learned my addr 172.21.15.196:0/119620715 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:46:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd923fff640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 0x7fd92419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd9237fe640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 0x7fd924197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 -- 172.21.15.196:0/119620715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 msgr2=0x7fd92419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 0x7fd92419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 -- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 msgr2=0x7fd924197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 0x7fd924197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 -- 172.21.15.196:0/119620715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92419f9d0 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd9237fe640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 0x7fd924197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd91affd640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c920 tx=0x7fd90c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c007c70 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92419fcc0 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c007e10 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c018820 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.821+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9241a01a0 con 0x7fd924102e00 2024-09-12T02:46:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.822+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd924108fa0 con 0x7fd924102e00 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.823+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd90c0189c0 con 0x7fd924102e00 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.824+0000 7fd9217fa640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.825+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6727+0+0 (secure 0 0 0) 0x7fd90c0ceb40 con 0x7fd924102e00 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.825+0000 7fd9237fe640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.825+0000 7fd9237fe640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 0x7fd9100a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd914006010 tx=0x7fd914032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:21.829+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd90c014030 con 0x7fd924102e00 2024-09-12T02:46:22.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:22.004+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd924062630 con 0x7fd9100a2b80 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:22 smithi196 ceph-mon[34587]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:22 smithi145 ceph-mon[34674]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:22.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:22 smithi046 ceph-mon[30951]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:46:23.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:23 smithi196 ceph-mon[34587]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:23.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:23 smithi196 ceph-mon[34587]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T02:46:23.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:23 smithi145 ceph-mon[34674]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:23.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:23 smithi145 ceph-mon[34674]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T02:46:23.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:23 smithi046 ceph-mon[30951]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:23.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:23 smithi046 ceph-mon[30951]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/2150501818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]': finished 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:25.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:25 smithi196 ceph-mon[34587]: from='client.? 172.21.15.196:0/2843680787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:46:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T02:46:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/2150501818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]': finished 2024-09-12T02:46:25.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:46:25.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:25.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:25 smithi145 ceph-mon[34674]: from='client.? 172.21.15.196:0/2843680787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/2150501818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]: dispatch 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2a6c3d9-b9e3-4985-b4b1-e1ac67272126"}]': finished 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:25 smithi046 ceph-mon[30951]: from='client.? 172.21.15.196:0/2843680787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:46:27.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:27 smithi145 ceph-mon[34674]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:27 smithi196 ceph-mon[34587]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:27.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:27 smithi046 ceph-mon[30951]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:29.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:29 smithi196 ceph-mon[34587]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:46:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:29 smithi145 ceph-mon[34674]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:46:29.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:29 smithi046 ceph-mon[30951]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:46:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:31 smithi145 ceph-mon[34674]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-12T02:46:31.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:31 smithi196 ceph-mon[34587]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-12T02:46:31.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:31 smithi046 ceph-mon[30951]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-12T02:46:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:33 smithi145 ceph-mon[34674]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:46:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:33 smithi196 ceph-mon[34587]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:46:33.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:33 smithi046 ceph-mon[30951]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:46:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:34 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:46:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:34 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:34 smithi196 ceph-mon[34587]: Deploying daemon osd.11 on smithi196 2024-09-12T02:46:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:34 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:46:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:34 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:34 smithi145 ceph-mon[34674]: Deploying daemon osd.11 on smithi196 2024-09-12T02:46:34.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:34 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:46:34.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:34 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:34.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:34 smithi046 ceph-mon[30951]: Deploying daemon osd.11 on smithi196 2024-09-12T02:46:35.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:35 smithi196 ceph-mon[34587]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:46:35.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:35 smithi145 ceph-mon[34674]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:46:35.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:35 smithi046 ceph-mon[30951]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:46:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:37 smithi145 ceph-mon[34674]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:46:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:37 smithi196 ceph-mon[34587]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:46:37.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:37 smithi046 ceph-mon[30951]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:46:39.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:39 smithi046 ceph-mon[30951]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:46:39.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:39 smithi145 ceph-mon[34674]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:46:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:39 smithi196 ceph-mon[34587]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:46:40.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:40.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:40.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:40.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:40.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:40.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:40.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:40.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:40.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:41.499 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:41 smithi196 ceph-mon[34587]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:41.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:41 smithi046 ceph-mon[30951]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:41 smithi145 ceph-mon[34674]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:43.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.156+0000 7fd9217fa640 1 -- 172.21.15.196:0/119620715 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd924062630 con 0x7fd9100a2b80 2024-09-12T02:46:43.157 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 msgr2=0x7fd9100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 0x7fd9100a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd914006010 tx=0x7fd914032d20 comp rx=0 tx=0).stop 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 msgr2=0x7fd9241982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c920 tx=0x7fd90c00cde0 comp rx=0 tx=0).stop 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 shutdown_connections 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.158+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9241037c0 0x7fd92419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd924102e00 0x7fd9241982f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:43.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 --2- 172.21.15.196:0/119620715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd924101c00 0x7fd924197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:43.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 >> 172.21.15.196:0/119620715 conn(0x7fd9240fd390 msgr2=0x7fd924106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:43.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 shutdown_connections 2024-09-12T02:46:43.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:43.159+0000 7fd929a8a640 1 -- 172.21.15.196:0/119620715 wait complete. 2024-09-12T02:46:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:43.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:43.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:43 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:43 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:43.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:43.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:43.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:43 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:43.898 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T02:46:43.900 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T02:46:43.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:46:44.169 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 -- 172.21.15.46:0/551044810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c1079c0 msgr2=0x7f2c7c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/551044810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c109db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009ca0 tx=0x7f2c7402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 -- 172.21.15.46:0/551044810 shutdown_connections 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/551044810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c109db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/551044810 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c105090 0x7f2c7c107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/551044810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c102760 0x7f2c7c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 -- 172.21.15.46:0/551044810 >> 172.21.15.46:0/551044810 conn(0x7f2c7c0fc2a0 msgr2=0x7f2c7c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 -- 172.21.15.46:0/551044810 shutdown_connections 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 -- 172.21.15.46:0/551044810 wait complete. 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.634+0000 7f2c81a7f640 1 Processor -- start 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 -- start start 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c102760 0x7f2c7c195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c1a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d780 con 0x7f2c7c105090 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d600 con 0x7f2c7c102760 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c81a7f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d900 con 0x7f2c7c1079c0 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50256/0 (socket says 172.21.15.46:50256) 2024-09-12T02:46:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 -- 172.21.15.46:0/4063666823 learned_addr learned my addr 172.21.15.46:0/4063666823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7bfff640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c1a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7b7fe640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c102760 0x7f2c7c195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 -- 172.21.15.46:0/4063666823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c1079c0 msgr2=0x7f2c7c1a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c1a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 -- 172.21.15.46:0/4063666823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c102760 msgr2=0x7f2c7c195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c102760 0x7f2c7c195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 -- 172.21.15.46:0/4063666823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c1a6040 con 0x7f2c7c105090 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c7affd640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9f0 tx=0x7f2c7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.635+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000cd80 con 0x7f2c7c105090 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.636+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c700054d0 con 0x7f2c7c105090 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.636+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1a6330 con 0x7f2c7c105090 2024-09-12T02:46:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.636+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70010490 con 0x7f2c7c105090 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.636+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a6870 con 0x7f2c7c105090 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.637+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c1094b0 con 0x7f2c7c105090 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.638+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c70005000 con 0x7f2c7c105090 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.638+0000 7f2c78ff9640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 0x7f2c640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.638+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f2c700cee70 con 0x7f2c7c105090 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.638+0000 7f2c7b7fe640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 0x7f2c640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.639+0000 7f2c7b7fe640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 0x7f2c640a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009b60 tx=0x7f2c6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:44.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.643+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c700052f0 con 0x7f2c7c105090 2024-09-12T02:46:44.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.819+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2c7c02d930 con 0x7f2c7c105090 2024-09-12T02:46:44.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.820+0000 7f2c78ff9640 1 -- 172.21.15.46:0/4063666823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f2c7009c3b0 con 0x7f2c7c105090 2024-09-12T02:46:44.820 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 msgr2=0x7f2c640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 0x7f2c640a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009b60 tx=0x7f2c6c005f90 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 msgr2=0x7f2c7c196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000e9f0 tx=0x7f2c7000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 shutdown_connections 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2c640a2b80 0x7f2c640a5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c7c1079c0 0x7f2c7c1a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c7c105090 0x7f2c7c196130 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 --2- 172.21.15.46:0/4063666823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c7c102760 0x7f2c7c195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 >> 172.21.15.46:0/4063666823 conn(0x7f2c7c0fc2a0 msgr2=0x7f2c7c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 shutdown_connections 2024-09-12T02:46:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:44.822+0000 7f2c81a7f640 1 -- 172.21.15.46:0/4063666823 wait complete. 2024-09-12T02:46:45.388 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:45 smithi196 ceph-mon[34587]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:45.388 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:45 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4063666823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:45.699 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:45 smithi046 ceph-mon[30951]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:45.699 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:45 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4063666823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:45.700 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 02:46:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T02:46:45.558+0000 7f9908589640 -1 osd.11 0 log_to_monitors true 2024-09-12T02:46:45.733 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109181,"num_in_osds":12,"osd_in_since":1726109184,"num_remapped_pgs":0} 2024-09-12T02:46:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:45 smithi145 ceph-mon[34674]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:45 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4063666823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:46 smithi196 ceph-mon[34587]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:46 smithi196 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:46.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:46:46.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:46 smithi046 ceph-mon[30951]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:46.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:46 smithi046 ceph-mon[30951]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:46 smithi145 ceph-mon[34674]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:46 smithi145 ceph-mon[34674]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:46:47.010 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- 172.21.15.46:0/1391227853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8102e00 msgr2=0x7fe0d8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 --2- 172.21.15.46:0/1391227853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8102e00 0x7fe0d8103280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009900 tx=0x7fe0c802a3f0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- 172.21.15.46:0/1391227853 shutdown_connections 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 --2- 172.21.15.46:0/1391227853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d81037c0 0x7fe0d810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 --2- 172.21.15.46:0/1391227853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8102e00 0x7fe0d8103280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 --2- 172.21.15.46:0/1391227853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d8101c00 0x7fe0d8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- 172.21.15.46:0/1391227853 >> 172.21.15.46:0/1391227853 conn(0x7fe0d80fd390 msgr2=0x7fe0d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- 172.21.15.46:0/1391227853 shutdown_connections 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- 172.21.15.46:0/1391227853 wait complete. 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 Processor -- start 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.519+0000 7fe0e044f640 1 -- start start 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 0x7fe0d8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 0x7fe0d819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810d710 con 0x7fe0d8101c00 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810d590 con 0x7fe0d8102e00 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0e044f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810d890 con 0x7fe0d81037c0 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50272/0 (socket says 172.21.15.46:50272) 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 -- 172.21.15.46:0/3916005309 learned_addr learned my addr 172.21.15.46:0/3916005309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0dd9ea640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 0x7fe0d8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 -- 172.21.15.46:0/3916005309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 msgr2=0x7fe0d819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de9ec640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 0x7fe0d819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 0x7fe0d819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 -- 172.21.15.46:0/3916005309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 msgr2=0x7fe0d8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 0x7fe0d8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 -- 172.21.15.46:0/3916005309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d819fa50 con 0x7fe0d8101c00 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de9ec640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 0x7fe0d819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0dd9ea640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 0x7fe0d8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0de1eb640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d770 tx=0x7fe0d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.520+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4017070 con 0x7fe0d8101c00 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.521+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d819fd40 con 0x7fe0d8101c00 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.521+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d4009c30 con 0x7fe0d8101c00 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.521+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4003d00 con 0x7fe0d8101c00 2024-09-12T02:46:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.521+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d81a01c0 con 0x7fe0d8101c00 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.522+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d81090c0 con 0x7fe0d8101c00 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.523+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe0d4003ea0 con 0x7fe0d8101c00 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.523+0000 7fe0cf7fe640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 0x7fe0c40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.524+0000 7fe0dd9ea640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 0x7fe0c40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.524+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7fe0d40ce8c0 con 0x7fe0d8101c00 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.524+0000 7fe0dd9ea640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 0x7fe0c40a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8199340 tx=0x7fe0c8032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.527+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0d409bde0 con 0x7fe0d8101c00 2024-09-12T02:46:47.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.701+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe0d802d7c0 con 0x7fe0d8101c00 2024-09-12T02:46:47.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:47.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: Detected new or changed devices on smithi196 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-12T02:46:47.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:47.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:47.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:47 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.702+0000 7fe0cf7fe640 1 -- 172.21.15.46:0/3916005309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fe0d409b6e0 con 0x7fe0d8101c00 2024-09-12T02:46:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 msgr2=0x7fe0c40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 0x7fe0c40a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8199340 tx=0x7fe0c8032c40 comp rx=0 tx=0).stop 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 msgr2=0x7fe0d8197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d770 tx=0x7fe0d400dc30 comp rx=0 tx=0).stop 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 shutdown_connections 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fe0c40a2bb0 0x7fe0c40a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0d81037c0 0x7fe0d819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d8102e00 0x7fe0d8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 --2- 172.21.15.46:0/3916005309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d8101c00 0x7fe0d8197e20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.704+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 >> 172.21.15.46:0/3916005309 conn(0x7fe0d80fd390 msgr2=0x7fe0d8106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.705+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 shutdown_connections 2024-09-12T02:46:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:47.705+0000 7fe0e044f640 1 -- 172.21.15.46:0/3916005309 wait complete. 2024-09-12T02:46:47.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T02:46:47.403+0000 7f990450c640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T02:46:47.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T02:46:47.416+0000 7f9900334640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: Detected new or changed devices on smithi196 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:47.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:47 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:46:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: Detected new or changed devices on smithi196 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:47.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:47 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:48.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:48 smithi046 ceph-mon[30951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:46:48.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:48 smithi046 ceph-mon[30951]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T02:46:48.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:48 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:48 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3916005309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:48.610 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109181,"num_in_osds":12,"osd_in_since":1726109184,"num_remapped_pgs":0} 2024-09-12T02:46:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:48 smithi196 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:46:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:48 smithi196 ceph-mon[34587]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T02:46:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:48 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:48 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3916005309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:48.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:48 smithi145 ceph-mon[34674]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-12T02:46:48.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:48 smithi145 ceph-mon[34674]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T02:46:48.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:48 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:48.762 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:48 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3916005309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:49.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:46:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: purged_snaps scrub starts 2024-09-12T02:46:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: purged_snaps scrub ok 2024-09-12T02:46:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] boot 2024-09-12T02:46:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:46:49.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:49 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:49.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: purged_snaps scrub starts 2024-09-12T02:46:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: purged_snaps scrub ok 2024-09-12T02:46:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] boot 2024-09-12T02:46:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:46:49.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:49 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: purged_snaps scrub starts 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: purged_snaps scrub ok 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: osd.11 [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] boot 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:46:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:49 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/3360594002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf74102e00 msgr2=0x7fbf74103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/3360594002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf74102e00 0x7fbf74103280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009900 tx=0x7fbf7002a410 comp rx=0 tx=0).stop 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/3360594002 shutdown_connections 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/3360594002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf741037c0 0x7fbf7410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/3360594002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf74102e00 0x7fbf74103280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/3360594002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/3360594002 >> 172.21.15.46:0/3360594002 conn(0x7fbf740fd390 msgr2=0x7fbf740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/3360594002 shutdown_connections 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.491+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/3360594002 wait complete. 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 Processor -- start 2024-09-12T02:46:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 -- start start 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf74102e00 0x7fbf740745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410d730 con 0x7fbf741037c0 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410d5b0 con 0x7fbf74102e00 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7c2b3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410d8b0 con 0x7fbf74101c00 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 -- 172.21.15.46:0/2821420518 learned_addr learned my addr 172.21.15.46:0/2821420518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 -- 172.21.15.46:0/2821420518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 msgr2=0x7fbf74075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a04f640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 -- 172.21.15.46:0/2821420518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf74102e00 msgr2=0x7fbf740745e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf74102e00 0x7fbf740745e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 -- 172.21.15.46:0/2821420518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740753e0 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a04f640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.492+0000 7fbf7a850640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00cce0 tx=0x7fbf6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.493+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00f450 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.493+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c00faa0 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.493+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c018b70 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.493+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a1f70 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.493+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a2420 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.494+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf74109050 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.495+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf6c018d10 con 0x7fbf741037c0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.496+0000 7fbf677fe640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.496+0000 7fbf7a04f640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.496+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbf6c0cf000 con 0x7fbf741037c0 2024-09-12T02:46:50.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.496+0000 7fbf7a04f640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 0x7fbf600a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf74102c60 tx=0x7fbf68005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:50.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.500+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf6c014030 con 0x7fbf741037c0 2024-09-12T02:46:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.677+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbf7402d970 con 0x7fbf741037c0 2024-09-12T02:46:50.677 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:50 smithi046 ceph-mon[30951]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:46:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.677+0000 7fbf677fe640 1 -- 172.21.15.46:0/2821420518 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7fbf6c09c400 con 0x7fbf741037c0 2024-09-12T02:46:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:46:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 msgr2=0x7fbf600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 0x7fbf600a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf74102c60 tx=0x7fbf68005e70 comp rx=0 tx=0).stop 2024-09-12T02:46:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 msgr2=0x7fbf74074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00cce0 tx=0x7fbf6c00a620 comp rx=0 tx=0).stop 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 shutdown_connections 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf741037c0 0x7fbf74074b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf74102e00 0x7fbf740745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 --2- 172.21.15.46:0/2821420518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf74101c00 0x7fbf74075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 >> 172.21.15.46:0/2821420518 conn(0x7fbf740fd390 msgr2=0x7fbf74106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 shutdown_connections 2024-09-12T02:46:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:50.680+0000 7fbf7c2b3640 1 -- 172.21.15.46:0/2821420518 wait complete. 2024-09-12T02:46:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:50 smithi196 ceph-mon[34587]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:46:50.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:50 smithi145 ceph-mon[34674]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:46:51.408 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726109208,"num_in_osds":12,"osd_in_since":1726109184,"num_remapped_pgs":0} 2024-09-12T02:46:51.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:46:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:51.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:51 smithi196 ceph-mon[34587]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:51.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:51 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2821420518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:51.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:51 smithi046 ceph-mon[30951]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:51.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:51 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2821420518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:51 smithi145 ceph-mon[34674]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:51 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2821420518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:46:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 -- 172.21.15.46:0/896537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc103650 msgr2=0x7f49fc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 --2- 172.21.15.46:0/896537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc103650 0x7f49fc10bb40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009900 tx=0x7f49f002a480 comp rx=0 tx=0).stop 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 -- 172.21.15.46:0/896537335 shutdown_connections 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 --2- 172.21.15.46:0/896537335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc10c160 0x7f49fc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 --2- 172.21.15.46:0/896537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc103650 0x7f49fc10bb40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 --2- 172.21.15.46:0/896537335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc102d30 0x7f49fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 -- 172.21.15.46:0/896537335 >> 172.21.15.46:0/896537335 conn(0x7f49fc0fc7d0 msgr2=0x7f49fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.111+0000 7f4a02562640 1 -- 172.21.15.46:0/896537335 shutdown_connections 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.112+0000 7f4a02562640 1 -- 172.21.15.46:0/896537335 wait complete. 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.112+0000 7f4a02562640 1 Processor -- start 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.112+0000 7f4a02562640 1 -- start start 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc103650 0x7f49fc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 0x7f49fc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc111ea0 con 0x7f49fc102d30 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc111d20 con 0x7f49fc103650 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a02562640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc112020 con 0x7f49fc10c160 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50308/0 (socket says 172.21.15.46:50308) 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 -- 172.21.15.46:0/2779009046 learned_addr learned my addr 172.21.15.46:0/2779009046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fb7fe640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc103650 0x7f49fc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a00aff640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 0x7f49fc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 -- 172.21.15.46:0/2779009046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 msgr2=0x7f49fc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 0x7f49fc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 -- 172.21.15.46:0/2779009046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc103650 msgr2=0x7f49fc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc103650 0x7f49fc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 -- 172.21.15.46:0/2779009046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc19d3b0 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f4a00aff640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 0x7f49fc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.113+0000 7f49fbfff640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b4a0 tx=0x7f49ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.114+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec014070 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.114+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49ec010ac0 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.114+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc19d6a0 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.114+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec017c30 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.114+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a4f80 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.115+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc06b070 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.116+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49ec017070 con 0x7f49fc102d30 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.116+0000 7f49f97fa640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 0x7f49e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.117+0000 7f49fb7fe640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 0x7f49e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:52.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.117+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f49ec0ce1f0 con 0x7f49fc102d30 2024-09-12T02:46:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.117+0000 7f49fb7fe640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 0x7f49e40a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005ab0 tx=0x7f49f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.121+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49ec09b5f0 con 0x7f49fc102d30 2024-09-12T02:46:52.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.298+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49fc1071d0 con 0x7f49fc102d30 2024-09-12T02:46:52.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.299+0000 7f49f97fa640 1 -- 172.21.15.46:0/2779009046 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18444 (secure 0 0 0) 0x7f49ec09aef0 con 0x7f49fc102d30 2024-09-12T02:46:52.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:46:52.300 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:46:49.408291+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 msgr2=0x7f49e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 0x7f49e40a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005ab0 tx=0x7f49f0005a20 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 msgr2=0x7f49fc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b4a0 tx=0x7f49ec00b960 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 shutdown_connections 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f49e40a2c00 0x7f49e40a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49fc10c160 0x7f49fc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f49fc103650 0x7f49fc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 --2- 172.21.15.46:0/2779009046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc102d30 0x7f49fc198400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 >> 172.21.15.46:0/2779009046 conn(0x7f49fc0fc7d0 msgr2=0x7f49fc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 shutdown_connections 2024-09-12T02:46:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:52.301+0000 7f4a02562640 1 -- 172.21.15.46:0/2779009046 wait complete. 2024-09-12T02:46:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:52 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2779009046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:46:52.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:52 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2779009046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:46:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:52 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2779009046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:46:53.106 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:42:56.661399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:46:53.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T02:46:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:53.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:53 smithi046 ceph-mon[30951]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:53.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:53 smithi196 ceph-mon[34587]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:53.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:53 smithi145 ceph-mon[34674]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 -- 172.21.15.46:0/3507930074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf101037a0 msgr2=0x7fbf1010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 --2- 172.21.15.46:0/3507930074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf101037a0 0x7fbf1010bc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009900 tx=0x7fbef802a410 comp rx=0 tx=0).stop 2024-09-12T02:46:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 -- 172.21.15.46:0/3507930074 shutdown_connections 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 --2- 172.21.15.46:0/3507930074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf1010c2f0 0x7fbf1010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 --2- 172.21.15.46:0/3507930074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf101037a0 0x7fbf1010bc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 --2- 172.21.15.46:0/3507930074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 -- 172.21.15.46:0/3507930074 >> 172.21.15.46:0/3507930074 conn(0x7fbf100fc7d0 msgr2=0x7fbf100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 -- 172.21.15.46:0/3507930074 shutdown_connections 2024-09-12T02:46:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.990+0000 7fbf14912640 1 -- 172.21.15.46:0/3507930074 wait complete. 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.991+0000 7fbf14912640 1 Processor -- start 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.991+0000 7fbf14912640 1 -- start start 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf10100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf101037a0 0x7fbf10100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10111fe0 con 0x7fbf1010c2f0 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10111e60 con 0x7fbf101037a0 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf14912640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10112160 con 0x7fbf10102df0 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50320/0 (socket says 172.21.15.46:50320) 2024-09-12T02:46:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 -- 172.21.15.46:0/2628395960 learned_addr learned my addr 172.21.15.46:0/2628395960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf07fff640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf101037a0 0x7fbf10100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0e59c640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf10100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 -- 172.21.15.46:0/2628395960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 msgr2=0x7fbf10100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf10100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 -- 172.21.15.46:0/2628395960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf101037a0 msgr2=0x7fbf10100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf101037a0 0x7fbf10100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 -- 172.21.15.46:0/2628395960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf101a6aa0 con 0x7fbf1010c2f0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0e59c640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf10100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf0ed9d640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800eaa0 tx=0x7fbf0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.992+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800cdb0 con 0x7fbf1010c2f0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.993+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf08005020 con 0x7fbf1010c2f0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.993+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101a6d90 con 0x7fbf1010c2f0 2024-09-12T02:46:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.993+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf080116d0 con 0x7fbf1010c2f0 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.993+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a7250 con 0x7fbf1010c2f0 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.994+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1006a790 con 0x7fbf1010c2f0 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.995+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf080051d0 con 0x7fbf1010c2f0 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.996+0000 7fbf077fe640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.996+0000 7fbf0e59c640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.996+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbf080cee60 con 0x7fbf1010c2f0 2024-09-12T02:46:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:53.996+0000 7fbf0e59c640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 0x7fbefc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009930 tx=0x7fbf00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.000+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf08016030 con 0x7fbf1010c2f0 2024-09-12T02:46:54.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.174+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf101072c0 con 0x7fbf1010c2f0 2024-09-12T02:46:54.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.174+0000 7fbf077fe640 1 -- 172.21.15.46:0/2628395960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fbf0809c1c0 con 0x7fbf1010c2f0 2024-09-12T02:46:54.175 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 msgr2=0x7fbefc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 0x7fbefc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009930 tx=0x7fbf00008040 comp rx=0 tx=0).stop 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 msgr2=0x7fbf101014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800eaa0 tx=0x7fbf0800ef60 comp rx=0 tx=0).stop 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 shutdown_connections 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf1010c2f0 0x7fbf101014d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf101037a0 0x7fbf10100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 --2- 172.21.15.46:0/2628395960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf10102df0 0x7fbf10100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.176+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 >> 172.21.15.46:0/2628395960 conn(0x7fbf100fc7d0 msgr2=0x7fbf100fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.177+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 shutdown_connections 2024-09-12T02:46:54.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:54.177+0000 7fbf14912640 1 -- 172.21.15.46:0/2628395960 wait complete. 2024-09-12T02:46:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:54 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2628395960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:46:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:54 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2628395960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:46:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:54 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2628395960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.b on smithi145 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.e on smithi145 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.h on smithi145 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.k on smithi145 2024-09-12T02:46:54.926 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-09-12T02:46:54.927 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-09-12T02:46:54.927 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-09-12T02:46:54.927 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-09-12T02:46:54.927 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-09-12T02:46:55.188 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:46:55.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:55 smithi196 ceph-mon[34587]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:55.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 -- 172.21.15.196:0/335663555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 msgr2=0x7f75b4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/335663555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4100ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f759c009900 tx=0x7f759c02a410 comp rx=0 tx=0).stop 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 -- 172.21.15.196:0/335663555 shutdown_connections 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/335663555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4102690 0x7f75b4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/335663555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4101cd0 0x7f75b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/335663555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4100ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 -- 172.21.15.196:0/335663555 >> 172.21.15.196:0/335663555 conn(0x7f75b40fc280 msgr2=0x7f75b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 -- 172.21.15.196:0/335663555 shutdown_connections 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.642+0000 7f75b8bc5640 1 -- 172.21.15.196:0/335663555 wait complete. 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 Processor -- start 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 -- start start 2024-09-12T02:46:55.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4101cd0 0x7f75b41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4102690 0x7f75b41035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75b410c6e0 con 0x7f75b4102690 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75b410c560 con 0x7f75b4101cd0 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b8bc5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75b410c860 con 0x7f75b4100ad0 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33122/0 (socket says 172.21.15.196:33122) 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 -- 172.21.15.196:0/2295342268 learned_addr learned my addr 172.21.15.196:0/2295342268 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b1d9b640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4101cd0 0x7f75b41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b2d9d640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4102690 0x7f75b41035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 -- 172.21.15.196:0/2295342268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4101cd0 msgr2=0x7f75b41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4101cd0 0x7f75b41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 -- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4102690 msgr2=0x7f75b41035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4102690 0x7f75b41035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.643+0000 7f75b259c640 1 -- 172.21.15.196:0/2295342268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b4103eb0 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75b259c640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f759c009230 tx=0x7f759c004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c03f030 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f759c003cb0 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b41a20f0 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c035070 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.644+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41a2670 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.645+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b41a2960 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.646+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f759c042030 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.647+0000 7f75ab7fe640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 0x7f75a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.648+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f759c0ef1a0 con 0x7f75b4100ad0 2024-09-12T02:46:55.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.648+0000 7f75b1d9b640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 0x7f75a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:55.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.648+0000 7f75b1d9b640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 0x7f75a00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b41047e0 tx=0x7f75a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:55.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.652+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f759c0bc5a0 con 0x7f75b4100ad0 2024-09-12T02:46:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:55 smithi046 ceph-mon[30951]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:55.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.825+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f75b406a850 con 0x7f75a00a2c00 2024-09-12T02:46:55.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.834+0000 7f75ab7fe640 1 -- 172.21.15.196:0/2295342268 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f75b406a850 con 0x7f75a00a2c00 2024-09-12T02:46:55.835 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 msgr2=0x7f75a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 0x7f75a00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b41047e0 tx=0x7f75a4009110 comp rx=0 tx=0).stop 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 msgr2=0x7f75b4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f759c009230 tx=0x7f759c004340 comp rx=0 tx=0).stop 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 shutdown_connections 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b4102690 0x7f75b41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75b4101cd0 0x7f75b41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f75a00a2c00 0x7f75a00a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 --2- 172.21.15.196:0/2295342268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75b4100ad0 0x7f75b4104a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:55.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 >> 172.21.15.196:0/2295342268 conn(0x7f75b40fc280 msgr2=0x7f75b406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:55.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 shutdown_connections 2024-09-12T02:46:55.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-12T02:46:55.836+0000 7f75b8bc5640 1 -- 172.21.15.196:0/2295342268 wait complete. 2024-09-12T02:46:55.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:55 smithi145 ceph-mon[34674]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:56.545 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T02:46:56.548 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T02:46:56.590 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T02:46:56.633 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T02:46:56.676 DEBUG:teuthology.orchestra.run.smithi145:mds.b> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T02:46:56.679 DEBUG:teuthology.orchestra.run.smithi145:mds.e> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T02:46:56.722 DEBUG:teuthology.orchestra.run.smithi145:mds.h> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T02:46:56.766 DEBUG:teuthology.orchestra.run.smithi145:mds.k> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T02:46:56.809 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T02:46:56.811 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T02:46:56.855 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T02:46:56.898 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T02:46:56.940 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T02:46:56.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:56.959 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:46:56.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:46:56.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:56.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:56 smithi196 ceph-mon[34587]: Deploying daemon mds.l on smithi196 2024-09-12T02:46:57.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:57.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:56 smithi145 ceph-mon[34674]: Deploying daemon mds.l on smithi196 2024-09-12T02:46:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:46:57.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:46:57.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:56 smithi046 ceph-mon[30951]: Deploying daemon mds.l on smithi196 2024-09-12T02:46:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.680+0000 7f0dde575640 1 -- 172.21.15.46:0/1842915385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 msgr2=0x7f0dd8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 --2- 172.21.15.46:0/1842915385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd8101200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009900 tx=0x7f0dc802a480 comp rx=0 tx=0).stop 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 -- 172.21.15.46:0/1842915385 shutdown_connections 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 --2- 172.21.15.46:0/1842915385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102ef0 0x7f0dd810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 --2- 172.21.15.46:0/1842915385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102570 0x7f0dd81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 --2- 172.21.15.46:0/1842915385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd8101200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 -- 172.21.15.46:0/1842915385 >> 172.21.15.46:0/1842915385 conn(0x7f0dd80fc7d0 msgr2=0x7f0dd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 -- 172.21.15.46:0/1842915385 shutdown_connections 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 -- 172.21.15.46:0/1842915385 wait complete. 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 Processor -- start 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.681+0000 7f0dde575640 1 -- start start 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102570 0x7f0dd80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 0x7f0dd80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8110e00 con 0x7f0dd8100e20 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8110c80 con 0x7f0dd8102ef0 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dde575640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8110f80 con 0x7f0dd8102570 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42486/0 (socket says 172.21.15.46:42486) 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 -- 172.21.15.46:0/3695452663 learned_addr learned my addr 172.21.15.46:0/3695452663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd77fe640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102570 0x7f0dd80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0ddcb12640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 0x7f0dd80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 -- 172.21.15.46:0/3695452663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102570 msgr2=0x7f0dd80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102570 0x7f0dd80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 -- 172.21.15.46:0/3695452663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 msgr2=0x7f0dd80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:57.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 0x7f0dd80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 -- 172.21.15.46:0/3695452663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd81a67f0 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0ddcb12640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 0x7f0dd80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd7fff640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006010 tx=0x7f0dc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.682+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc803f070 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.683+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc8003aa0 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.683+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd81a6a80 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.683+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8035030 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.683+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81a6f60 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.684+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd8101200 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.685+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0dc8042070 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.685+0000 7f0dd57fa640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 0x7f0dc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.685+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0dc80ef580 con 0x7f0dd8100e20 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.685+0000 7f0dd77fe640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 0x7f0dc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:46:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.686+0000 7f0dd77fe640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 0x7f0dc00a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005e10 tx=0x7f0dcc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:46:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.690+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dc80bc980 con 0x7f0dd8100e20 2024-09-12T02:46:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.905+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd810a0a0 con 0x7f0dd8100e20 2024-09-12T02:46:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.906+0000 7f0dd57fa640 1 -- 172.21.15.46:0/3695452663 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f0dc80dc340 con 0x7f0dd8100e20 2024-09-12T02:46:57.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:46:57.906 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"btime":"2024-09-12T02:39:49:303263+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T02:46:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-09-12T02:46:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 msgr2=0x7f0dc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 0x7f0dc00a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005e10 tx=0x7f0dcc005da0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 msgr2=0x7f0dd80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006010 tx=0x7f0dc80042f0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 shutdown_connections 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd8102ef0 0x7f0dd80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0dc00a2b80 0x7f0dc00a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd8102570 0x7f0dd80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 --2- 172.21.15.46:0/3695452663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd8100e20 0x7f0dd80fcc80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 >> 172.21.15.46:0/3695452663 conn(0x7f0dd80fc7d0 msgr2=0x7f0dd806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 shutdown_connections 2024-09-12T02:46:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:46:57.908+0000 7f0dde575640 1 -- 172.21.15.46:0/3695452663 wait complete. 2024-09-12T02:46:58.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:58 smithi196 ceph-mon[34587]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:58.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:58 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3695452663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:46:58.460 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 systemd[1]: Starting Ceph mds.l for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:46:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:58 smithi046 ceph-mon[30951]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:58.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:58 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3695452663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:46:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:58 smithi145 ceph-mon[34674]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:46:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:58 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3695452663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:46:58.720 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 podman[63774]: 2024-09-12 02:46:58.362160381 +0000 UTC m=+0.015638765 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:46:58.721 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 podman[63774]: 2024-09-12 02:46:58.482336831 +0000 UTC m=+0.135815243 container create 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T02:46:59.220 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 podman[63774]: 2024-09-12 02:46:58.743343457 +0000 UTC m=+0.396821840 container init 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:46:59.221 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 podman[63774]: 2024-09-12 02:46:58.746636982 +0000 UTC m=+0.400115367 container start 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T02:46:59.221 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 ceph-mds[63793]: -- 172.21.15.196:0/1880123373 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a233cc3c20 con 0x55a233ca0800 2024-09-12T02:46:59.221 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: starting mds.l at 2024-09-12T02:46:59.221 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 bash[63774]: 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c 2024-09-12T02:46:59.221 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 02:46:58 smithi196 systemd[1]: Started Ceph mds.l for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6832/1048929159,v1:172.21.15.196:6833/1048929159] up:boot 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: fsmap 1 up:standby 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:46:59 smithi046 ceph-mon[30951]: Deploying daemon mds.k on smithi145 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6832/1048929159,v1:172.21.15.196:6833/1048929159] up:boot 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: fsmap 1 up:standby 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.035 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:00.035 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:00.035 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:00.035 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:46:59 smithi145 ceph-mon[34674]: Deploying daemon mds.k on smithi145 2024-09-12T02:47:00.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6832/1048929159,v1:172.21.15.196:6833/1048929159] up:boot 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: fsmap 1 up:standby 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:00.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:46:59 smithi196 ceph-mon[34587]: Deploying daemon mds.k on smithi145 2024-09-12T02:47:00.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:01.110 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:00 smithi145 systemd[1]: Starting Ceph mds.k for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 -- 172.21.15.46:0/24477821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184101de0 msgr2=0x7f61841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 --2- 172.21.15.46:0/24477821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184101de0 0x7f61841021c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6174009900 tx=0x7f617402a410 comp rx=0 tx=0).stop 2024-09-12T02:47:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 -- 172.21.15.46:0/24477821 shutdown_connections 2024-09-12T02:47:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 --2- 172.21.15.46:0/24477821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184106f10 0x7f618410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 --2- 172.21.15.46:0/24477821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 0x7f61841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 --2- 172.21.15.46:0/24477821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184101de0 0x7f61841021c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 -- 172.21.15.46:0/24477821 >> 172.21.15.46:0/24477821 conn(0x7f61840fd8c0 msgr2=0x7f61840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 -- 172.21.15.46:0/24477821 shutdown_connections 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.416+0000 7f61888fe640 1 -- 172.21.15.46:0/24477821 wait complete. 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 Processor -- start 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 -- start start 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 0x7f6184198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 0x7f6184198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6184111e70 con 0x7f6184106f10 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6184111cf0 con 0x7f6184102790 2024-09-12T02:47:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f61888fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6184111ff0 con 0x7f6184101de0 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f6182d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f6182d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42500/0 (socket says 172.21.15.46:42500) 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.417+0000 7f6182d9d640 1 -- 172.21.15.46:0/2769621304 learned_addr learned my addr 172.21.15.46:0/2769621304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6181d9b640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 0x7f6184198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f618259c640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 0x7f6184198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 -- 172.21.15.46:0/2769621304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 msgr2=0x7f6184198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 0x7f6184198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 -- 172.21.15.46:0/2769621304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 msgr2=0x7f6184198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 0x7f6184198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 -- 172.21.15.46:0/2769621304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618419d300 con 0x7f6184106f10 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f618259c640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 0x7f6184198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f6182d9d640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f617c00cce0 tx=0x7f617c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:01.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00f450 con 0x7f6184106f10 2024-09-12T02:47:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617c00faa0 con 0x7f6184106f10 2024-09-12T02:47:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618419d5f0 con 0x7f6184106f10 2024-09-12T02:47:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c018b70 con 0x7f6184106f10 2024-09-12T02:47:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.418+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841a4ed0 con 0x7f6184106f10 2024-09-12T02:47:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.419+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61841021c0 con 0x7f6184106f10 2024-09-12T02:47:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.420+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f617c018d10 con 0x7f6184106f10 2024-09-12T02:47:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.421+0000 7f617b7fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 0x7f61700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.421+0000 7f618259c640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 0x7f61700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.421+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f617c0ceef0 con 0x7f6184106f10 2024-09-12T02:47:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.422+0000 7f618259c640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 0x7f61700a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ab0 tx=0x7f6174000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.426+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f617c014030 con 0x7f6184106f10 2024-09-12T02:47:01.483 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 podman[63380]: 2024-09-12 02:47:01.051551148 +0000 UTC m=+0.016125272 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:01.483 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 podman[63380]: 2024-09-12 02:47:01.179901031 +0000 UTC m=+0.144475199 container create 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T02:47:01.483 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 podman[63380]: 2024-09-12 02:47:01.483071573 +0000 UTC m=+0.447645682 container init 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:47:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.637+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f618402d7c0 con 0x7f6184106f10 2024-09-12T02:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.637+0000 7f617b7fe640 1 -- 172.21.15.46:0/2769621304 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f617c09c2f0 con 0x7f6184106f10 2024-09-12T02:47:01.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:01.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"btime":"2024-09-12T02:46:58:774888+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 msgr2=0x7f61700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 0x7f61700a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ab0 tx=0x7f6174000bc0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 msgr2=0x7f618419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f617c00cce0 tx=0x7f617c00a620 comp rx=0 tx=0).stop 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 shutdown_connections 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6184106f10 0x7f618419cc20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184102790 0x7f6184198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f61700a2c00 0x7f61700a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 --2- 172.21.15.46:0/2769621304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6184101de0 0x7f6184198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 >> 172.21.15.46:0/2769621304 conn(0x7f61840fd8c0 msgr2=0x7f61840fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 shutdown_connections 2024-09-12T02:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:01.641+0000 7f61888fe640 1 -- 172.21.15.46:0/2769621304 wait complete. 2024-09-12T02:47:01.783 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 podman[63380]: 2024-09-12 02:47:01.486185295 +0000 UTC m=+0.450759407 container start 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:47:01.783 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mds[63399]: -- 172.21.15.145:0/130780934 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596efbd1c20 con 0x5596efbaf000 2024-09-12T02:47:01.783 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k[63395]: starting mds.k at 2024-09-12T02:47:01.783 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 bash[63380]: 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a 2024-09-12T02:47:01.783 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 02:47:01 smithi145 systemd[1]: Started Ceph mds.k for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2769621304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:01 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2769621304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:02.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:02.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:01 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:02.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2769621304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:02.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:01 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:02.922 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:02 smithi145 ceph-mon[34674]: Deploying daemon mds.j on smithi046 2024-09-12T02:47:02.922 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:02 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6832/1820698401,v1:172.21.15.145:6833/1820698401] up:boot 2024-09-12T02:47:02.922 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:02 smithi145 ceph-mon[34674]: fsmap 2 up:standby 2024-09-12T02:47:02.922 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:02 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:47:03.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:02 smithi046 ceph-mon[30951]: Deploying daemon mds.j on smithi046 2024-09-12T02:47:03.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:02 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6832/1820698401,v1:172.21.15.145:6833/1820698401] up:boot 2024-09-12T02:47:03.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:02 smithi046 ceph-mon[30951]: fsmap 2 up:standby 2024-09-12T02:47:03.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:02 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:47:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:02 smithi196 ceph-mon[34587]: Deploying daemon mds.j on smithi046 2024-09-12T02:47:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:02 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6832/1820698401,v1:172.21.15.145:6833/1820698401] up:boot 2024-09-12T02:47:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:02 smithi196 ceph-mon[34587]: fsmap 2 up:standby 2024-09-12T02:47:03.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:02 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:47:03.973 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:03 smithi046 ceph-mon[30951]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:03 smithi145 ceph-mon[34674]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:04.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:03 smithi196 ceph-mon[34587]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:04.282 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 systemd[1]: Starting Ceph mds.j for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:04.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:04.721 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 podman[70450]: 2024-09-12 02:47:04.275245055 +0000 UTC m=+0.016305019 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:04.721 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 podman[70450]: 2024-09-12 02:47:04.40030945 +0000 UTC m=+0.141369473 container create 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T02:47:04.721 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 podman[70450]: 2024-09-12 02:47:04.720469894 +0000 UTC m=+0.461529858 container init 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T02:47:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:05.032 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 podman[70450]: 2024-09-12 02:47:04.72371659 +0000 UTC m=+0.464776555 container start 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T02:47:05.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 ceph-mds[70500]: -- 172.21.15.46:0/1187483245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4046a3860 con 0x55b404680800 2024-09-12T02:47:05.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j[70483]: starting mds.j at 2024-09-12T02:47:05.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 bash[70450]: 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 2024-09-12T02:47:05.033 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 02:47:04 smithi046 systemd[1]: Started Ceph mds.j for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 -- 172.21.15.46:0/3575454760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 msgr2=0x7efec010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 --2- 172.21.15.46:0/3575454760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec010ea90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ca0 tx=0x7efeb802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 -- 172.21.15.46:0/3575454760 shutdown_connections 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 --2- 172.21.15.46:0/3575454760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec010ea90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 --2- 172.21.15.46:0/3575454760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 0x7efec01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 --2- 172.21.15.46:0/3575454760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 0x7efec01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 -- 172.21.15.46:0/3575454760 >> 172.21.15.46:0/3575454760 conn(0x7efec00fd8c0 msgr2=0x7efec00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 -- 172.21.15.46:0/3575454760 shutdown_connections 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.457+0000 7efec6c96640 1 -- 172.21.15.46:0/3575454760 wait complete. 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 Processor -- start 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 -- start start 2024-09-12T02:47:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 0x7efec0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 0x7efec0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efec0111e20 con 0x7efec0106f10 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efec0111ca0 con 0x7efec0102790 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec6c96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efec0111fa0 con 0x7efec0101de0 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.458+0000 7efec5233640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46474/0 (socket says 172.21.15.46:46474) 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 -- 172.21.15.46:0/1314059547 learned_addr learned my addr 172.21.15.46:0/1314059547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec4a32640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 0x7efec0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efebffff640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 0x7efec0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 -- 172.21.15.46:0/1314059547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 msgr2=0x7efec0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 0x7efec0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 -- 172.21.15.46:0/1314059547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 msgr2=0x7efec0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 0x7efec0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 -- 172.21.15.46:0/1314059547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec019d2d0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec5233640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009dc0 tx=0x7efeb8032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8003cf0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb80044f0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec019d560 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8030bf0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.459+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec01a4ea0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.460+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec01021c0 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.462+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efeb8004040 con 0x7efec0106f10 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.462+0000 7efebdffb640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 0x7efeb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.462+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7efeb80f0150 con 0x7efec0106f10 2024-09-12T02:47:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.462+0000 7efec4a32640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 0x7efeb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.463+0000 7efec4a32640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 0x7efeb00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.467+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efeb80bd550 con 0x7efec0106f10 2024-09-12T02:47:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.684+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efec010a0f0 con 0x7efec0106f10 2024-09-12T02:47:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.685+0000 7efebdffb640 1 -- 172.21.15.46:0/1314059547 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7efeb80bce50 con 0x7efec0106f10 2024-09-12T02:47:05.686 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:05.686 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"btime":"2024-09-12T02:47:04:789535+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:47:05.686 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 msgr2=0x7efeb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 0x7efeb00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb400b390 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 msgr2=0x7efec019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009dc0 tx=0x7efeb8032ff0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 shutdown_connections 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec0106f10 0x7efec019cbf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efec0102790 0x7efec0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efeb00a2b80 0x7efeb00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 --2- 172.21.15.46:0/1314059547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec0101de0 0x7efec0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 >> 172.21.15.46:0/1314059547 conn(0x7efec00fd8c0 msgr2=0x7efec00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 shutdown_connections 2024-09-12T02:47:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:05.687+0000 7efec6c96640 1 -- 172.21.15.46:0/1314059547 wait complete. 2024-09-12T02:47:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.46:6834/2542241089,v1:172.21.15.46:6835/2542241089] up:boot 2024-09-12T02:47:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: fsmap 3 up:standby 2024-09-12T02:47:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: Deploying daemon mds.d on smithi046 2024-09-12T02:47:05.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:05 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1314059547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:06.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:06.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.46:6834/2542241089,v1:172.21.15.46:6835/2542241089] up:boot 2024-09-12T02:47:06.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: fsmap 3 up:standby 2024-09-12T02:47:06.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: Deploying daemon mds.d on smithi046 2024-09-12T02:47:06.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:05 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1314059547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.46:6834/2542241089,v1:172.21.15.46:6835/2542241089] up:boot 2024-09-12T02:47:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: fsmap 3 up:standby 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: Deploying daemon mds.d on smithi046 2024-09-12T02:47:06.111 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:05 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1314059547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:08.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:07 smithi046 ceph-mon[30951]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:08.033 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:07 smithi046 systemd[1]: Starting Ceph mds.d for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:07 smithi145 ceph-mon[34674]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:07 smithi196 ceph-mon[34587]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:08.379 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 podman[70864]: 2024-09-12 02:47:08.016482704 +0000 UTC m=+0.016497987 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:08.379 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 podman[70864]: 2024-09-12 02:47:08.142838941 +0000 UTC m=+0.142854257 container create 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:47:08.379 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 podman[70864]: 2024-09-12 02:47:08.379316785 +0000 UTC m=+0.379332059 container init 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T02:47:08.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:08.782 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 podman[70864]: 2024-09-12 02:47:08.382477953 +0000 UTC m=+0.382493226 container start 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:47:08.783 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 ceph-mds[70883]: -- 172.21.15.46:0/412751155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5cbc95860 con 0x55d5cbc72800 2024-09-12T02:47:08.783 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d[70879]: starting mds.d at 2024-09-12T02:47:08.783 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 bash[70864]: 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 2024-09-12T02:47:08.783 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 02:47:08 smithi046 systemd[1]: Started Ceph mds.d for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 -- 172.21.15.46:0/4041692854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 msgr2=0x7f86781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 --2- 172.21.15.46:0/4041692854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86781021c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8668009900 tx=0x7f866802a410 comp rx=0 tx=0).stop 2024-09-12T02:47:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 -- 172.21.15.46:0/4041692854 shutdown_connections 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 --2- 172.21.15.46:0/4041692854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 0x7f867810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 --2- 172.21.15.46:0/4041692854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 0x7f86781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 --2- 172.21.15.46:0/4041692854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86781021c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 -- 172.21.15.46:0/4041692854 >> 172.21.15.46:0/4041692854 conn(0x7f86780fd8c0 msgr2=0x7f86780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 -- 172.21.15.46:0/4041692854 shutdown_connections 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 -- 172.21.15.46:0/4041692854 wait complete. 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.261+0000 7f867d42a640 1 Processor -- start 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 -- start start 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 0x7f8678074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 0x7f8678074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8678111e40 con 0x7f8678101de0 2024-09-12T02:47:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8678111cc0 con 0x7f8678106f10 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f867d42a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8678111fc0 con 0x7f8678102790 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46508/0 (socket says 172.21.15.46:46508) 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 -- 172.21.15.46:0/1821871111 learned_addr learned my addr 172.21.15.46:0/1821871111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f86777fe640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 0x7f8678074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f866ffff640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 0x7f8678074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 -- 172.21.15.46:0/1821871111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 msgr2=0x7f8678074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 0x7f8678074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 -- 172.21.15.46:0/1821871111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 msgr2=0x7f8678074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 0x7f8678074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.262+0000 7f8676ffd640 1 -- 172.21.15.46:0/1821871111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86781a6840 con 0x7f8678101de0 2024-09-12T02:47:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f8676ffd640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f8668004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866803f030 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86781a6ad0 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86680035f0 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668035070 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.263+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781a7000 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.264+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86781021c0 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.265+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8668003b10 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.266+0000 7f8674ff9640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 0x7f86640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.266+0000 7f866ffff640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 0x7f86640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.266+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f86680ef950 con 0x7f8678101de0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.266+0000 7f866ffff640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 0x7f86640a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8678075f80 tx=0x7f8660009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:09.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.273+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86680bccb0 con 0x7f8678101de0 2024-09-12T02:47:09.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.485+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f867810a0f0 con 0x7f8678101de0 2024-09-12T02:47:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.486+0000 7f8674ff9640 1 -- 172.21.15.46:0/1821871111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f86680bc5b0 con 0x7f8678101de0 2024-09-12T02:47:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:09.487 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"btime":"2024-09-12T02:47:08:798151+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:47:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 msgr2=0x7f86640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 0x7f86640a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8678075f80 tx=0x7f8660009110 comp rx=0 tx=0).stop 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 msgr2=0x7f86780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f8668004270 comp rx=0 tx=0).stop 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 shutdown_connections 2024-09-12T02:47:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8678106f10 0x7f8678074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86640a2b80 0x7f86640a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8678102790 0x7f8678074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 --2- 172.21.15.46:0/1821871111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8678101de0 0x7f86780744f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 >> 172.21.15.46:0/1821871111 conn(0x7f86780fd8c0 msgr2=0x7f86780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 shutdown_connections 2024-09-12T02:47:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:09.488+0000 7f867d42a640 1 -- 172.21.15.46:0/1821871111 wait complete. 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: Deploying daemon mds.e on smithi145 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.46:6836/4040861117,v1:172.21.15.46:6837/4040861117] up:boot 2024-09-12T02:47:09.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: fsmap 4 up:standby 2024-09-12T02:47:09.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:47:09.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:09 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1821871111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:09.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:09.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: Deploying daemon mds.e on smithi145 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.46:6836/4040861117,v1:172.21.15.46:6837/4040861117] up:boot 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: fsmap 4 up:standby 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:47:09.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:09 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1821871111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: Deploying daemon mds.e on smithi145 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.46:6836/4040861117,v1:172.21.15.46:6837/4040861117] up:boot 2024-09-12T02:47:09.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: fsmap 4 up:standby 2024-09-12T02:47:09.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:47:09.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:09 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1821871111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:10.823 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:10 smithi145 systemd[1]: Starting Ceph mds.e for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:11.110 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:10 smithi145 podman[63691]: 2024-09-12 02:47:10.725321473 +0000 UTC m=+0.015786578 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:11.110 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:10 smithi145 podman[63691]: 2024-09-12 02:47:10.852680051 +0000 UTC m=+0.143145152 container create 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:47:11.553 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 podman[63691]: 2024-09-12 02:47:11.113958539 +0000 UTC m=+0.404423638 container init 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:47:11.553 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 podman[63691]: 2024-09-12 02:47:11.116749173 +0000 UTC m=+0.407214271 container start 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:47:11.553 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mds[63710]: -- 172.21.15.145:0/967711252 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583f7e53860 con 0x5583f7e31000 2024-09-12T02:47:11.553 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e[63706]: starting mds.e at 2024-09-12T02:47:11.554 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 bash[63691]: 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 2024-09-12T02:47:11.554 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 02:47:11 smithi145 systemd[1]: Started Ceph mds.e for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:11 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:11.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:11 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:12.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:11 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:12.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:12.845 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:12 smithi046 ceph-mon[30951]: Deploying daemon mds.f on smithi196 2024-09-12T02:47:12.845 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:12 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6834/198433993,v1:172.21.15.145:6835/198433993] up:boot 2024-09-12T02:47:12.845 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:12 smithi046 ceph-mon[30951]: fsmap 5 up:standby 2024-09-12T02:47:12.846 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:12 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:47:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:12 smithi145 ceph-mon[34674]: Deploying daemon mds.f on smithi196 2024-09-12T02:47:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:12 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6834/198433993,v1:172.21.15.145:6835/198433993] up:boot 2024-09-12T02:47:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:12 smithi145 ceph-mon[34674]: fsmap 5 up:standby 2024-09-12T02:47:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:12 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:47:12.881 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:12 smithi196 ceph-mon[34587]: Deploying daemon mds.f on smithi196 2024-09-12T02:47:12.881 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:12 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6834/198433993,v1:172.21.15.145:6835/198433993] up:boot 2024-09-12T02:47:12.881 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:12 smithi196 ceph-mon[34587]: fsmap 5 up:standby 2024-09-12T02:47:12.881 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:12 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:47:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.927+0000 7f273081a640 1 -- 172.21.15.46:0/2661629132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 msgr2=0x7f272c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.927+0000 7f273081a640 1 --2- 172.21.15.46:0/2661629132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f271c009900 tx=0x7f271c02a480 comp rx=0 tx=0).stop 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 -- 172.21.15.46:0/2661629132 shutdown_connections 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 --2- 172.21.15.46:0/2661629132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102ef0 0x7f272c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 --2- 172.21.15.46:0/2661629132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102570 0x7f272c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 --2- 172.21.15.46:0/2661629132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 -- 172.21.15.46:0/2661629132 >> 172.21.15.46:0/2661629132 conn(0x7f272c0fc7d0 msgr2=0x7f272c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 -- 172.21.15.46:0/2661629132 shutdown_connections 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 -- 172.21.15.46:0/2661629132 wait complete. 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.928+0000 7f273081a640 1 Processor -- start 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 -- start start 2024-09-12T02:47:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102570 0x7f272c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102ef0 0x7f272c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f272c110ed0 con 0x7f272c100e20 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f272c110d50 con 0x7f272c102ef0 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f273081a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f272c111050 con 0x7f272c102570 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46526/0 (socket says 172.21.15.46:46526) 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 -- 172.21.15.46:0/1774090651 learned_addr learned my addr 172.21.15.46:0/1774090651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f2721d9b640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102570 0x7f272c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272ad9d640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102ef0 0x7f272c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 -- 172.21.15.46:0/1774090651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102570 msgr2=0x7f272c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102570 0x7f272c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 -- 172.21.15.46:0/1774090651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102ef0 msgr2=0x7f272c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102ef0 0x7f272c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 -- 172.21.15.46:0/1774090651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c1a6a00 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.929+0000 7f272a59c640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f271c006010 tx=0x7f271c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.930+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c03f070 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.930+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f271c003aa0 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.930+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c1a6c90 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.930+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c035030 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.930+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1a7170 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.931+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c101200 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.932+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f271c042070 con 0x7f272c100e20 2024-09-12T02:47:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.932+0000 7f2723fff640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.933+0000 7f2721d9b640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.933+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f271c0ef5f0 con 0x7f272c100e20 2024-09-12T02:47:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.933+0000 7f2721d9b640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 0x7f27180a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f272c0fe710 tx=0x7f2714009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:12.937+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f271c0bc950 con 0x7f272c100e20 2024-09-12T02:47:13.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.147+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f272c10a0a0 con 0x7f272c100e20 2024-09-12T02:47:13.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.148+0000 7f2723fff640 1 -- 172.21.15.46:0/1774090651 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f271c0bc250 con 0x7f272c100e20 2024-09-12T02:47:13.148 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:13.148 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"btime":"2024-09-12T02:47:11:548008+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:47:13.149 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 msgr2=0x7f27180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 0x7f27180a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f272c0fe710 tx=0x7f2714009110 comp rx=0 tx=0).stop 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 msgr2=0x7f272c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f271c006010 tx=0x7f271c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 shutdown_connections 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f272c102ef0 0x7f272c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:13.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f272c102570 0x7f272c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:13.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 --2- 172.21.15.46:0/1774090651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f272c100e20 0x7f272c0fcc80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:13.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 >> 172.21.15.46:0/1774090651 conn(0x7f272c0fc7d0 msgr2=0x7f272c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:13.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 shutdown_connections 2024-09-12T02:47:13.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:13.150+0000 7f273081a640 1 -- 172.21.15.46:0/1774090651 wait complete. 2024-09-12T02:47:13.560 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 podman[64075]: 2024-09-12 02:47:13.486642406 +0000 UTC m=+0.127711864 container create 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T02:47:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:13 smithi046 ceph-mon[30951]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:13 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1774090651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:13.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:13 smithi196 ceph-mon[34587]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:13.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:13 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1774090651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:13.848 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 podman[64075]: 2024-09-12 02:47:13.739607092 +0000 UTC m=+0.380676554 container init 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T02:47:13.848 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 podman[64075]: 2024-09-12 02:47:13.742344832 +0000 UTC m=+0.383414290 container start 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T02:47:13.848 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 ceph-mds[64094]: -- 172.21.15.196:0/3795764751 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597fac2d860 con 0x5597fac0b000 2024-09-12T02:47:13.848 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: starting mds.f at 2024-09-12T02:47:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:13 smithi145 ceph-mon[34674]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:13 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1774090651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:14.220 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 bash[64075]: 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 2024-09-12T02:47:14.220 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 02:47:13 smithi196 systemd[1]: Started Ceph mds.f for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:15.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: Deploying daemon mds.c on smithi196 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6834/3561512892,v1:172.21.15.196:6835/3561512892] up:boot 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: fsmap 6 up:standby 2024-09-12T02:47:15.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:14 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: Deploying daemon mds.c on smithi196 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6834/3561512892,v1:172.21.15.196:6835/3561512892] up:boot 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: fsmap 6 up:standby 2024-09-12T02:47:15.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:14 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:15.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:15.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: Deploying daemon mds.c on smithi196 2024-09-12T02:47:15.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6834/3561512892,v1:172.21.15.196:6835/3561512892] up:boot 2024-09-12T02:47:15.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: fsmap 6 up:standby 2024-09-12T02:47:15.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:14 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:47:15.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:16.124 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:15 smithi196 ceph-mon[34587]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:16.271 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:15 smithi046 ceph-mon[30951]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:15 smithi145 ceph-mon[34674]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:16.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.651+0000 7f7807f2d640 1 -- 172.21.15.46:0/1067413900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 msgr2=0x7f7800103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.651+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1067413900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800103110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009900 tx=0x7f77fc02a480 comp rx=0 tx=0).stop 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 -- 172.21.15.46:0/1067413900 shutdown_connections 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1067413900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f780010c160 0x7f780010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1067413900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800103650 0x7f780010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1067413900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800103110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 -- 172.21.15.46:0/1067413900 >> 172.21.15.46:0/1067413900 conn(0x7f78000fc790 msgr2=0x7f78000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 -- 172.21.15.46:0/1067413900 shutdown_connections 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 -- 172.21.15.46:0/1067413900 wait complete. 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 Processor -- start 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 -- start start 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.652+0000 7f7807f2d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800103650 0x7f78001988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 0x7f780019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7800111e10 con 0x7f7800102d30 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7800111c90 con 0x7f780010c160 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7800111f90 con 0x7f7800103650 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39190/0 (socket says 172.21.15.46:39190) 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 -- 172.21.15.46:0/1150449346 learned_addr learned my addr 172.21.15.46:0/1150449346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f78054c8640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800103650 0x7f78001988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f78064ca640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 0x7f780019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 -- 172.21.15.46:0/1150449346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800103650 msgr2=0x7f78001988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800103650 0x7f78001988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 -- 172.21.15.46:0/1150449346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 msgr2=0x7f780019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 0x7f780019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 -- 172.21.15.46:0/1150449346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780019d310 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f78064ca640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 0x7f780019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7805cc9640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77fc005940 tx=0x7f77fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc03f030 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780019d5a0 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77fc003aa0 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc035070 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.653+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78001a4ee0 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.654+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780006b070 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.656+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f77fc042030 con 0x7f7800102d30 2024-09-12T02:47:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.657+0000 7f77f2ffd640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 0x7f77ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.657+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f77fc0ef530 con 0x7f7800102d30 2024-09-12T02:47:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.657+0000 7f78054c8640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 0x7f77ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.657+0000 7f78054c8640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 0x7f77ec0a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f78000fee00 tx=0x7f77f400b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:16.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.660+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77fc0bc930 con 0x7f7800102d30 2024-09-12T02:47:16.720 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 02:47:16 smithi196 systemd[1]: Started Ceph mds.c for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.870+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7800107200 con 0x7f7800102d30 2024-09-12T02:47:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.871+0000 7f77f2ffd640 1 -- 172.21.15.46:0/1150449346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7f77fc0bc230 con 0x7f7800102d30 2024-09-12T02:47:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:16.873 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"btime":"2024-09-12T02:47:14:559472+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-12T02:47:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-09-12T02:47:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 msgr2=0x7f77ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 0x7f77ec0a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f78000fee00 tx=0x7f77f400b360 comp rx=0 tx=0).stop 2024-09-12T02:47:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 msgr2=0x7f7800198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77fc005940 tx=0x7f77fc004270 comp rx=0 tx=0).stop 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 shutdown_connections 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f780010c160 0x7f780019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f77ec0a2b80 0x7f77ec0a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800103650 0x7f78001988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 --2- 172.21.15.46:0/1150449346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7800102d30 0x7f7800198360 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 >> 172.21.15.46:0/1150449346 conn(0x7f78000fc790 msgr2=0x7f78000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 shutdown_connections 2024-09-12T02:47:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:16.874+0000 7f7807f2d640 1 -- 172.21.15.46:0/1150449346 wait complete. 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: Deploying daemon mds.h on smithi145 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1150449346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:boot 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: fsmap 7 up:standby 2024-09-12T02:47:17.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:17 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:47:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: Deploying daemon mds.h on smithi145 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1150449346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:boot 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: fsmap 7 up:standby 2024-09-12T02:47:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:17 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: Deploying daemon mds.h on smithi145 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1150449346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:boot 2024-09-12T02:47:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: fsmap 7 up:standby 2024-09-12T02:47:18.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:17 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:47:18.682 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:18 smithi145 systemd[1]: Starting Ceph mds.h for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:19.057 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:18 smithi145 podman[63992]: 2024-09-12 02:47:18.797129421 +0000 UTC m=+0.015785533 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:19.057 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:18 smithi145 podman[63992]: 2024-09-12 02:47:18.925982355 +0000 UTC m=+0.144638454 container create 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 podman[63992]: 2024-09-12 02:47:19.188100502 +0000 UTC m=+0.406756610 container init 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 podman[63992]: 2024-09-12 02:47:19.191463681 +0000 UTC m=+0.410119790 container start 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mds[64012]: -- 172.21.15.145:0/2553119995 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55896d527860 con 0x55896d505000 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: starting mds.h at 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 bash[63992]: 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 2024-09-12T02:47:19.315 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 02:47:19 smithi145 systemd[1]: Started Ceph mds.h for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:19.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:19.834 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:19 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:19.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:19.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:19 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:19.977 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:19.977 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:19.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:19.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:19.978 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:19 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- 172.21.15.46:0/2866212633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e7010e480 msgr2=0x7f0e70110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 --2- 172.21.15.46:0/2866212633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e7010e480 0x7f0e70110870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009ca0 tx=0x7f0e6802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- 172.21.15.46:0/2866212633 shutdown_connections 2024-09-12T02:47:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 --2- 172.21.15.46:0/2866212633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e7010e480 0x7f0e70110870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 --2- 172.21.15.46:0/2866212633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e70101280 0x7f0e7010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 --2- 172.21.15.46:0/2866212633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e70100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- 172.21.15.46:0/2866212633 >> 172.21.15.46:0/2866212633 conn(0x7f0e700fc7a0 msgr2=0x7f0e700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- 172.21.15.46:0/2866212633 shutdown_connections 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- 172.21.15.46:0/2866212633 wait complete. 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 Processor -- start 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.409+0000 7f0e7810e640 1 -- start start 2024-09-12T02:47:20.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e7019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 0x7f0e7019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e70114020 con 0x7f0e70101280 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e70113ea0 con 0x7f0e7010e480 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e7810e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701141a0 con 0x7f0e701008d0 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:20.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39210/0 (socket says 172.21.15.46:39210) 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 -- 172.21.15.46:0/1510453444 learned_addr learned my addr 172.21.15.46:0/1510453444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e75eaa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e7019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e766ab640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 0x7f0e7019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 -- 172.21.15.46:0/1510453444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 msgr2=0x7f0e7019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e7019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 -- 172.21.15.46:0/1510453444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 msgr2=0x7f0e7019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 0x7f0e7019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 -- 172.21.15.46:0/1510453444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7019f530 con 0x7f0e70101280 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e75eaa640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e7019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e766ab640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 0x7f0e7019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.410+0000 7f0e756a9640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000e9f0 tx=0x7f0e6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.411+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000cd80 con 0x7f0e70101280 2024-09-12T02:47:20.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.411+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e600108e0 con 0x7f0e70101280 2024-09-12T02:47:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.411+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7019f820 con 0x7f0e70101280 2024-09-12T02:47:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.411+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60018920 con 0x7f0e70101280 2024-09-12T02:47:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.411+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e701a7100 con 0x7f0e70101280 2024-09-12T02:47:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.412+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e70100cb0 con 0x7f0e70101280 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.413+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e60010a90 con 0x7f0e70101280 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.414+0000 7f0e66ffd640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 0x7f0e5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.414+0000 7f0e75eaa640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 0x7f0e5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.414+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0e600ceed0 con 0x7f0e70101280 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.414+0000 7f0e75eaa640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 0x7f0e5c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009fc0 tx=0x7f0e6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.418+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e6009bbe0 con 0x7f0e70101280 2024-09-12T02:47:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.636+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e70109080 con 0x7f0e70101280 2024-09-12T02:47:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.636+0000 7f0e66ffd640 1 -- 172.21.15.46:0/1510453444 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7f0e6009fee0 con 0x7f0e70101280 2024-09-12T02:47:20.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:20.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"btime":"2024-09-12T02:47:19:580486+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T02:47:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-09-12T02:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 msgr2=0x7f0e5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 0x7f0e5c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009fc0 tx=0x7f0e6c005f90 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 msgr2=0x7f0e7019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000e9f0 tx=0x7f0e6000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 shutdown_connections 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e7010e480 0x7f0e7019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e70101280 0x7f0e7019aac0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0e5c0a2b80 0x7f0e5c0a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 --2- 172.21.15.46:0/1510453444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e701008d0 0x7f0e7019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 >> 172.21.15.46:0/1510453444 conn(0x7f0e700fc7a0 msgr2=0x7f0e700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 shutdown_connections 2024-09-12T02:47:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:20.639+0000 7f0e7810e640 1 -- 172.21.15.46:0/1510453444 wait complete. 2024-09-12T02:47:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:20 smithi145 ceph-mon[34674]: Deploying daemon mds.i on smithi196 2024-09-12T02:47:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:20 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6836/2556040092,v1:172.21.15.145:6837/2556040092] up:boot 2024-09-12T02:47:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:20 smithi145 ceph-mon[34674]: fsmap 8 up:standby 2024-09-12T02:47:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:20 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:47:20.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:20 smithi196 ceph-mon[34587]: Deploying daemon mds.i on smithi196 2024-09-12T02:47:20.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:20 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6836/2556040092,v1:172.21.15.145:6837/2556040092] up:boot 2024-09-12T02:47:20.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:20 smithi196 ceph-mon[34587]: fsmap 8 up:standby 2024-09-12T02:47:20.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:20 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:47:20.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:20 smithi046 ceph-mon[30951]: Deploying daemon mds.i on smithi196 2024-09-12T02:47:20.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:20 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6836/2556040092,v1:172.21.15.145:6837/2556040092] up:boot 2024-09-12T02:47:20.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:20 smithi046 ceph-mon[30951]: fsmap 8 up:standby 2024-09-12T02:47:20.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:20 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:47:21.470 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 systemd[1]: Starting Ceph mds.i for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:21.825 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:21 smithi196 ceph-mon[34587]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:21.826 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:21 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1510453444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:21.826 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 podman[64691]: 2024-09-12 02:47:21.463378121 +0000 UTC m=+0.015977011 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:21.826 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 podman[64691]: 2024-09-12 02:47:21.581300333 +0000 UTC m=+0.133899219 container create d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:47:21.826 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 podman[64691]: 2024-09-12 02:47:21.825923144 +0000 UTC m=+0.378522029 container init d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T02:47:21.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:21 smithi145 ceph-mon[34674]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:21.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:21 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1510453444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:21 smithi046 ceph-mon[30951]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:21 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1510453444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:22.220 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 podman[64691]: 2024-09-12 02:47:21.828747607 +0000 UTC m=+0.381346493 container start d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:47:22.220 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 ceph-mds[64711]: -- 172.21.15.196:0/1242947087 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555abdee9860 con 0x555abdec7400 2024-09-12T02:47:22.220 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: starting mds.i at 2024-09-12T02:47:22.221 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 bash[64691]: d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e 2024-09-12T02:47:22.221 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:21 smithi196 systemd[1]: Started Ceph mds.i for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:23.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: Deploying daemon mds.g on smithi046 2024-09-12T02:47:23.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:boot 2024-09-12T02:47:23.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: fsmap 9 up:standby 2024-09-12T02:47:23.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:22 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:47:23.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:23.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: Deploying daemon mds.g on smithi046 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:boot 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: fsmap 9 up:standby 2024-09-12T02:47:23.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:22 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:47:23.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: Deploying daemon mds.g on smithi046 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:boot 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: fsmap 9 up:standby 2024-09-12T02:47:23.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:22 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:47:23.686 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:24.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:24 smithi046 ceph-mon[30951]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:24.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:24 smithi145 ceph-mon[34674]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 -- 172.21.15.46:0/2209987786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 msgr2=0x7fcc0c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 --2- 172.21.15.46:0/2209987786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c10ea90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ca0 tx=0x7fcc0402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 -- 172.21.15.46:0/2209987786 shutdown_connections 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 --2- 172.21.15.46:0/2209987786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c10ea90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 --2- 172.21.15.46:0/2209987786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 0x7fcc0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 --2- 172.21.15.46:0/2209987786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 0x7fcc0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 -- 172.21.15.46:0/2209987786 >> 172.21.15.46:0/2209987786 conn(0x7fcc0c0fd8c0 msgr2=0x7fcc0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 -- 172.21.15.46:0/2209987786 shutdown_connections 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.340+0000 7fcc12e18640 1 -- 172.21.15.46:0/2209987786 wait complete. 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 Processor -- start 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 -- start start 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 0x7fcc0c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 0x7fcc0c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c111e40 con 0x7fcc0c106f10 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c111cc0 con 0x7fcc0c102790 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc12e18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c111fc0 con 0x7fcc0c101de0 2024-09-12T02:47:24.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc113b5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc113b5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59028/0 (socket says 172.21.15.46:59028) 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc113b5640 1 -- 172.21.15.46:0/1714509908 learned_addr learned my addr 172.21.15.46:0/1714509908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc10bb4640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 0x7fcc0c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc0bfff640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 0x7fcc0c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc113b5640 1 -- 172.21.15.46:0/1714509908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 msgr2=0x7fcc0c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.341+0000 7fcc113b5640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 0x7fcc0c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc113b5640 1 -- 172.21.15.46:0/1714509908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 msgr2=0x7fcc0c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc113b5640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 0x7fcc0c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc113b5640 1 -- 172.21.15.46:0/1714509908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c19d3b0 con 0x7fcc0c106f10 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc113b5640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009dc0 tx=0x7fcc04003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004040 con 0x7fcc0c106f10 2024-09-12T02:47:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c19d640 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc04004690 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04030c90 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.342+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c1a4f80 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.343+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c1021c0 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.344+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc040041e0 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.345+0000 7fcc09ffb640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.345+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fcc040f0380 con 0x7fcc0c106f10 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.345+0000 7fcc10bb4640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.345+0000 7fcc10bb4640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009990 tx=0x7fcc00009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:24.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.349+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc040bd6e0 con 0x7fcc0c106f10 2024-09-12T02:47:24.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:24 smithi196 ceph-mon[34587]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.575+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0c10a150 con 0x7fcc0c106f10 2024-09-12T02:47:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.575+0000 7fcc09ffb640 1 -- 172.21.15.46:0/1714509908 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7fcc040bcfe0 con 0x7fcc0c106f10 2024-09-12T02:47:24.577 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:24.578 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"btime":"2024-09-12T02:47:22:591561+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:47:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-09-12T02:47:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 msgr2=0x7fcbfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009990 tx=0x7fcc00009110 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 msgr2=0x7fcc0c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009dc0 tx=0x7fcc04003ba0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 shutdown_connections 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc0c106f10 0x7fcc0c19ccd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c102790 0x7fcc0c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 --2- 172.21.15.46:0/1714509908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc0c101de0 0x7fcc0c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 >> 172.21.15.46:0/1714509908 conn(0x7fcc0c0fd8c0 msgr2=0x7fcc0c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 shutdown_connections 2024-09-12T02:47:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:24.578+0000 7fcc12e18640 1 -- 172.21.15.46:0/1714509908 wait complete. 2024-09-12T02:47:25.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:25 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1714509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:25.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:25 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1714509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:25.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:25 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1714509908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:26.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:26.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: Deploying daemon mds.a on smithi046 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.46:6838/2433411125,v1:172.21.15.46:6839/2433411125] up:boot 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: fsmap 10 up:standby 2024-09-12T02:47:26.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:26 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:47:26.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:26.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: Deploying daemon mds.a on smithi046 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.46:6838/2433411125,v1:172.21.15.46:6839/2433411125] up:boot 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: fsmap 10 up:standby 2024-09-12T02:47:26.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:26 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:26.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:26.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: Deploying daemon mds.a on smithi046 2024-09-12T02:47:26.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.46:6838/2433411125,v1:172.21.15.46:6839/2433411125] up:boot 2024-09-12T02:47:26.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: fsmap 10 up:standby 2024-09-12T02:47:26.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:26 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:47:27.783 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:27 smithi046 systemd[1]: Starting Ceph mds.a for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T02:47:27.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:28.142 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:27 smithi046 podman[72049]: 2024-09-12 02:47:27.822010595 +0000 UTC m=+0.016336341 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:28.142 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:27 smithi046 podman[72049]: 2024-09-12 02:47:27.937710642 +0000 UTC m=+0.132036370 container create 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T02:47:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:28.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:28 smithi196 ceph-mon[34587]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:28.487 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:28 smithi046 ceph-mon[30951]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:28.487 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 podman[72049]: 2024-09-12 02:47:28.218707643 +0000 UTC m=+0.413033383 container init 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:47:28.487 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 podman[72049]: 2024-09-12 02:47:28.221778481 +0000 UTC m=+0.416104209 container start 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T02:47:28.487 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 ceph-mds[72112]: -- 172.21.15.46:0/1546828527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efa3d2b860 con 0x55efa3d09400 2024-09-12T02:47:28.488 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a[72102]: starting mds.a at 2024-09-12T02:47:28.488 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 bash[72049]: 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 2024-09-12T02:47:28.488 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 02:47:28 smithi046 systemd[1]: Started Ceph mds.a for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:28 smithi145 ceph-mon[34674]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 -- 172.21.15.46:0/3146612519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 msgr2=0x7f02e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 --2- 172.21.15.46:0/3146612519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e41067e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc02a410 comp rx=0 tx=0).stop 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 -- 172.21.15.46:0/3146612519 shutdown_connections 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 --2- 172.21.15.46:0/3146612519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4106f10 0x7f02e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 --2- 172.21.15.46:0/3146612519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e41067e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 --2- 172.21.15.46:0/3146612519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4101de0 0x7f02e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 -- 172.21.15.46:0/3146612519 >> 172.21.15.46:0/3146612519 conn(0x7f02e40fd880 msgr2=0x7f02e40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 -- 172.21.15.46:0/3146612519 shutdown_connections 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.020+0000 7f02e965e640 1 -- 172.21.15.46:0/3146612519 wait complete. 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 Processor -- start 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 -- start start 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4101de0 0x7f02e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4106f10 0x7f02e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4111e40 con 0x7f02e4102790 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4111cc0 con 0x7f02e4106f10 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e965e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4111fc0 con 0x7f02e4101de0 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.021+0000 7f02e27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59090/0 (socket says 172.21.15.46:59090) 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 -- 172.21.15.46:0/2045870862 learned_addr learned my addr 172.21.15.46:0/2045870862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e37fe640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4106f10 0x7f02e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e2ffd640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4101de0 0x7f02e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 -- 172.21.15.46:0/2045870862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4101de0 msgr2=0x7f02e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4101de0 0x7f02e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 -- 172.21.15.46:0/2045870862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4106f10 msgr2=0x7f02e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4106f10 0x7f02e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 -- 172.21.15.46:0/2045870862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e41a6880 con 0x7f02e4102790 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e27fc640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005940 tx=0x7f02cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc03f030 con 0x7f02e4102790 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02cc003aa0 con 0x7f02e4102790 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e41a6b10 con 0x7f02e4102790 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc035070 con 0x7f02e4102790 2024-09-12T02:47:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.022+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e41a6ff0 con 0x7f02e4102790 2024-09-12T02:47:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.023+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e41021c0 con 0x7f02e4102790 2024-09-12T02:47:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.024+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f02cc042030 con 0x7f02e4102790 2024-09-12T02:47:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.025+0000 7f02dbfff640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 0x7f02d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.025+0000 7f02e2ffd640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 0x7f02d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.025+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f02cc0ef3f0 con 0x7f02e4102790 2024-09-12T02:47:29.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.026+0000 7f02e2ffd640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 0x7f02d00a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:29.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.030+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02cc0bc750 con 0x7f02e4102790 2024-09-12T02:47:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.241+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02e410a0f0 con 0x7f02e4102790 2024-09-12T02:47:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.241+0000 7f02dbfff640 1 -- 172.21.15.46:0/2045870862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9704 (secure 0 0 0) 0x7f02cc0bc050 con 0x7f02e4102790 2024-09-12T02:47:29.243 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:29.243 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"btime":"2024-09-12T02:47:29:142163+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:47:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 12 2024-09-12T02:47:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 msgr2=0x7f02d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 0x7f02d00a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4005850 comp rx=0 tx=0).stop 2024-09-12T02:47:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 msgr2=0x7f02e4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005940 tx=0x7f02cc004270 comp rx=0 tx=0).stop 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 shutdown_connections 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4106f10 0x7f02e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e4102790 0x7f02e4074a30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f02d00a2b80 0x7f02d00a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 --2- 172.21.15.46:0/2045870862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e4101de0 0x7f02e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 >> 172.21.15.46:0/2045870862 conn(0x7f02e40fd880 msgr2=0x7f02e40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 shutdown_connections 2024-09-12T02:47:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:29.244+0000 7f02e965e640 1 -- 172.21.15.46:0/2045870862 wait complete. 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:29.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:29.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: Deploying daemon mds.b on smithi145 2024-09-12T02:47:29.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.46:6840/1897348158,v1:172.21.15.46:6841/1897348158] up:boot 2024-09-12T02:47:29.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: fsmap 11 up:standby 2024-09-12T02:47:29.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:47:29.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:29 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2045870862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: Deploying daemon mds.b on smithi145 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.46:6840/1897348158,v1:172.21.15.46:6841/1897348158] up:boot 2024-09-12T02:47:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: fsmap 11 up:standby 2024-09-12T02:47:29.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:47:29.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:29 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2045870862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:29.787 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:29.787 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.787 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.787 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: Deploying daemon mds.b on smithi145 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.46:6840/1897348158,v1:172.21.15.46:6841/1897348158] up:boot 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: fsmap 11 up:standby 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:47:29.788 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:29 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2045870862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:30.716 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:30 smithi145 podman[64298]: 2024-09-12 02:47:30.593198928 +0000 UTC m=+0.015857292 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:47:30.716 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:30 smithi145 podman[64298]: 2024-09-12 02:47:30.715189626 +0000 UTC m=+0.137848046 container create 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:47:31.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 podman[64298]: 2024-09-12 02:47:31.009772051 +0000 UTC m=+0.432430420 container init 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:47:31.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 podman[64298]: 2024-09-12 02:47:31.01259119 +0000 UTC m=+0.435249551 container start 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:47:31.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 ceph-mds[64317]: -- 172.21.15.145:0/3512577712 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558eb2d57860 con 0x558eb2d34800 2024-09-12T02:47:31.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: starting mds.b at 2024-09-12T02:47:31.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 bash[64298]: 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 2024-09-12T02:47:31.361 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 02:47:31 smithi145 systemd[1]: Started Ceph mds.b for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T02:47:32.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:32.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:boot 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: fsmap 12 up:standby 2024-09-12T02:47:32.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:32 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:47:32.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:32.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:boot 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: fsmap 12 up:standby 2024-09-12T02:47:32.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:32 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:47:32.488 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:boot 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: fsmap 12 up:standby 2024-09-12T02:47:32.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:32 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:47:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 -- 172.21.15.46:0/2884118167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc101de0 msgr2=0x7f6ccc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 --2- 172.21.15.46:0/2884118167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc101de0 0x7f6ccc1021c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009ca0 tx=0x7f6cbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 -- 172.21.15.46:0/2884118167 shutdown_connections 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 --2- 172.21.15.46:0/2884118167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc106f10 0x7f6ccc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 --2- 172.21.15.46:0/2884118167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 0x7f6ccc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 --2- 172.21.15.46:0/2884118167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc101de0 0x7f6ccc1021c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 -- 172.21.15.46:0/2884118167 >> 172.21.15.46:0/2884118167 conn(0x7f6ccc0fd8c0 msgr2=0x7f6ccc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 -- 172.21.15.46:0/2884118167 shutdown_connections 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.024+0000 7f6cd1422640 1 -- 172.21.15.46:0/2884118167 wait complete. 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 Processor -- start 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 -- start start 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc101de0 0x7f6ccc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 0x7f6ccc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc111e20 con 0x7f6ccc106f10 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc111ca0 con 0x7f6ccc102790 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6cd1422640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc111fa0 con 0x7f6ccc101de0 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.025+0000 7f6ccb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc101de0 0x7f6ccc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59116/0 (socket says 172.21.15.46:59116) 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 -- 172.21.15.46:0/528515357 learned_addr learned my addr 172.21.15.46:0/528515357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cca7fc640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 0x7f6ccc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 -- 172.21.15.46:0/528515357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc101de0 msgr2=0x7f6ccc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc101de0 0x7f6ccc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 -- 172.21.15.46:0/528515357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 msgr2=0x7f6ccc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 0x7f6ccc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 -- 172.21.15.46:0/528515357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc19d340 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6ccb7fe640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d750 tx=0x7f6cc400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4017070 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cc40054d0 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc19d630 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004c60 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.026+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1a4f10 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.027+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc1021c0 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.028+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc4004e00 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.029+0000 7f6cc3fff640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 0x7f6cb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.029+0000 7f6ccaffd640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 0x7f6cb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.029+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6cc40ce430 con 0x7f6ccc106f10 2024-09-12T02:47:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.030+0000 7f6ccaffd640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 0x7f6cb80a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005e30 tx=0x7f6cbc031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.033+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cc409b1e0 con 0x7f6ccc106f10 2024-09-12T02:47:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.254+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ccc10a150 con 0x7f6ccc106f10 2024-09-12T02:47:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.254+0000 7f6cc3fff640 1 -- 172.21.15.46:0/528515357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f6cc409f4e0 con 0x7f6ccc106f10 2024-09-12T02:47:33.256 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:33.256 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"btime":"2024-09-12T02:47:31:383068+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:47:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-09-12T02:47:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.257+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 msgr2=0x7f6cb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 0x7f6cb80a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005e30 tx=0x7f6cbc031000 comp rx=0 tx=0).stop 2024-09-12T02:47:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 msgr2=0x7f6ccc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d750 tx=0x7f6cc400dc10 comp rx=0 tx=0).stop 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 shutdown_connections 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ccc106f10 0x7f6ccc19cc60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ccc102790 0x7f6ccc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6cb80a2b80 0x7f6cb80a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 --2- 172.21.15.46:0/528515357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ccc101de0 0x7f6ccc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 >> 172.21.15.46:0/528515357 conn(0x7f6ccc0fd8c0 msgr2=0x7f6ccc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 shutdown_connections 2024-09-12T02:47:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:33.258+0000 7f6cd1422640 1 -- 172.21.15.46:0/528515357 wait complete. 2024-09-12T02:47:33.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:33.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:33.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:33.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:33.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:33.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/528515357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:33.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:33 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/528515357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:34.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:33 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/528515357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:34.251 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- 172.21.15.46:0/1604409733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c4101280 msgr2=0x7f38c410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 --2- 172.21.15.46:0/1604409733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c4101280 0x7f38c410df40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009900 tx=0x7f38b802a410 comp rx=0 tx=0).stop 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- 172.21.15.46:0/1604409733 shutdown_connections 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 --2- 172.21.15.46:0/1604409733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c410e480 0x7f38c4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 --2- 172.21.15.46:0/1604409733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c4101280 0x7f38c410df40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 --2- 172.21.15.46:0/1604409733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c41008d0 0x7f38c4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- 172.21.15.46:0/1604409733 >> 172.21.15.46:0/1604409733 conn(0x7f38c40fc7a0 msgr2=0x7f38c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- 172.21.15.46:0/1604409733 shutdown_connections 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- 172.21.15.46:0/1604409733 wait complete. 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 Processor -- start 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.799+0000 7f38ca287640 1 -- start start 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 0x7f38c419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 0x7f38c419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4114060 con 0x7f38c41008d0 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4113ee0 con 0x7f38c4101280 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38ca287640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41141e0 con 0x7f38c410e480 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50394/0 (socket says 172.21.15.46:50394) 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 -- 172.21.15.46:0/4229461043 learned_addr learned my addr 172.21.15.46:0/4229461043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c8824640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 0x7f38c419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c37fe640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 0x7f38c419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 -- 172.21.15.46:0/4229461043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 msgr2=0x7f38c419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 0x7f38c419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 -- 172.21.15.46:0/4229461043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 msgr2=0x7f38c419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 0x7f38c419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 -- 172.21.15.46:0/4229461043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c419f5d0 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c37fe640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 0x7f38c419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c8824640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 0x7f38c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.800+0000 7f38c3fff640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f38b400e9f0 tx=0x7f38b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.801+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400cd80 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.801+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b40054d0 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.801+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c419f8c0 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.801+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4010490 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.801+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a71a0 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.802+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4100cb0 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.803+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38b4005020 con 0x7f38c41008d0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.803+0000 7f38c17fa640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 0x7f38ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.804+0000 7f38c37fe640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 0x7f38ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.804+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f38b40cefb0 con 0x7f38c41008d0 2024-09-12T02:47:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.804+0000 7f38c37fe640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 0x7f38ac0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38b8031440 tx=0x7f38b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.808+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38b409c3b0 con 0x7f38c41008d0 2024-09-12T02:47:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.987+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38c4109610 con 0x7f38c41008d0 2024-09-12T02:47:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.988+0000 7f38c17fa640 1 -- 172.21.15.46:0/4229461043 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f38b409bcb0 con 0x7f38c41008d0 2024-09-12T02:47:34.989 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:34.989 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-12T02:47:34.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T02:47:34.990 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-12T02:47:34.990 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-12T02:47:34.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T02:47:34.990 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 msgr2=0x7f38ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 0x7f38ac0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38b8031440 tx=0x7f38b8005a20 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 msgr2=0x7f38c419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f38b400e9f0 tx=0x7f38b400edf0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 shutdown_connections 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f38ac0a2b80 0x7f38ac0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38c410e480 0x7f38c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38c4101280 0x7f38c419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 --2- 172.21.15.46:0/4229461043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c41008d0 0x7f38c419a620 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.990+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 >> 172.21.15.46:0/4229461043 conn(0x7f38c40fc7a0 msgr2=0x7f38c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.991+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 shutdown_connections 2024-09-12T02:47:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:34.991+0000 7f38ca287640 1 -- 172.21.15.46:0/4229461043 wait complete. 2024-09-12T02:47:35.185 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:35.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:35 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4229461043' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:35.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:35 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4229461043' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:47:35.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:35.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:47:35.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:35 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4229461043' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:47:35.705 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T02:47:35.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T02:47:36.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:36 smithi046 ceph-mon[30951]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:36 smithi145 ceph-mon[34674]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:36 smithi196 ceph-mon[34587]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 -- 172.21.15.46:0/41547194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 msgr2=0x7fb54810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 --2- 172.21.15.46:0/41547194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54810da40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb540009ca0 tx=0x7fb54002a950 comp rx=0 tx=0).stop 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 -- 172.21.15.46:0/41547194 shutdown_connections 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 --2- 172.21.15.46:0/41547194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54810da40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 --2- 172.21.15.46:0/41547194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 0x7fb5481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 --2- 172.21.15.46:0/41547194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 -- 172.21.15.46:0/41547194 >> 172.21.15.46:0/41547194 conn(0x7fb5480fc7d0 msgr2=0x7fb5480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 -- 172.21.15.46:0/41547194 shutdown_connections 2024-09-12T02:47:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.119+0000 7fb54dd16640 1 -- 172.21.15.46:0/41547194 wait complete. 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 Processor -- start 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 -- start start 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 0x7fb5481987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb548110e30 con 0x7fb548102ef0 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb548110cb0 con 0x7fb548102570 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb54dd16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb548110fb0 con 0x7fb548100e20 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50408/0 (socket says 172.21.15.46:50408) 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 -- 172.21.15.46:0/853536506 learned_addr learned my addr 172.21.15.46:0/853536506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb5477fe640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb53ffff640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 0x7fb5481987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 -- 172.21.15.46:0/853536506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 msgr2=0x7fb548198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 -- 172.21.15.46:0/853536506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 msgr2=0x7fb5481987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 0x7fb5481987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 -- 172.21.15.46:0/853536506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54819d230 con 0x7fb548102ef0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb5477fe640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.120+0000 7fb547fff640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb54002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.121+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540034040 con 0x7fb548102ef0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.121+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54819d4c0 con 0x7fb548102ef0 2024-09-12T02:47:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.121+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb540003e00 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.121+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540045480 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.121+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5481a4e00 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.122+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5481021b0 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.123+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb540037020 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.124+0000 7fb5457fa640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 0x7fb5340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.124+0000 7fb5477fe640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 0x7fb5340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.124+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb5400efd30 con 0x7fb548102ef0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.124+0000 7fb5477fe640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 0x7fb5340a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb53800abe0 tx=0x7fb53800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.128+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5400bd130 con 0x7fb548102ef0 2024-09-12T02:47:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:37.311+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb54810a0a0 con 0x7fb548102ef0 2024-09-12T02:47:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.045+0000 7fb5457fa640 1 -- 172.21.15.46:0/853536506 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fb5400bca30 con 0x7fb548102ef0 2024-09-12T02:47:38.047 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 msgr2=0x7fb5340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 0x7fb5340a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb53800abe0 tx=0x7fb53800a5c0 comp rx=0 tx=0).stop 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 msgr2=0x7fb54819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb54002f710 comp rx=0 tx=0).stop 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 shutdown_connections 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb548102ef0 0x7fb54819cb50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb548102570 0x7fb5481987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb5340a2b80 0x7fb5340a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:38.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 --2- 172.21.15.46:0/853536506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb548100e20 0x7fb548198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 >> 172.21.15.46:0/853536506 conn(0x7fb5480fc7d0 msgr2=0x7fb54806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 shutdown_connections 2024-09-12T02:47:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:38.048+0000 7fb54dd16640 1 -- 172.21.15.46:0/853536506 wait complete. 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:38.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:38.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:38.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:47:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:47:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:47:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:47:38.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T02:47:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:39 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:47:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:39 smithi145 ceph-mon[34674]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:47:39.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:39 smithi145 ceph-mon[34674]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:47:39.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:39 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:47:39.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:39 smithi046 ceph-mon[30951]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:47:39.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:39 smithi046 ceph-mon[30951]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:47:39.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:39 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/853536506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:47:39.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:39 smithi196 ceph-mon[34587]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:47:39.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:39 smithi196 ceph-mon[34587]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 -- 172.21.15.46:0/3532278106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564106f10 msgr2=0x7fd56410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/3532278106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564106f10 0x7fd56410ea90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009ca0 tx=0x7fd55c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 -- 172.21.15.46:0/3532278106 shutdown_connections 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/3532278106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564106f10 0x7fd56410ea90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/3532278106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd564102790 0x7fd5641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/3532278106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd564101de0 0x7fd5641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 -- 172.21.15.46:0/3532278106 >> 172.21.15.46:0/3532278106 conn(0x7fd5640fd8c0 msgr2=0x7fd5640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:39.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 -- 172.21.15.46:0/3532278106 shutdown_connections 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.463+0000 7fd56a4bd640 1 -- 172.21.15.46:0/3532278106 wait complete. 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 Processor -- start 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 -- start start 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd564102790 0x7fd564198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd564106f10 0x7fd56419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111e20 con 0x7fd564101de0 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111ca0 con 0x7fd564102790 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd56a4bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111fa0 con 0x7fd564106f10 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50424/0 (socket says 172.21.15.46:50424) 2024-09-12T02:47:39.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 -- 172.21.15.46:0/2313601127 learned_addr learned my addr 172.21.15.46:0/2313601127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 -- 172.21.15.46:0/2313601127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd564106f10 msgr2=0x7fd56419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd564106f10 0x7fd56419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 -- 172.21.15.46:0/2313601127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd564102790 msgr2=0x7fd564198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd564102790 0x7fd564198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 -- 172.21.15.46:0/2313601127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56419d2b0 con 0x7fd564101de0 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.464+0000 7fd563fff640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c890 tx=0x7fd55400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.465+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554004220 con 0x7fd564101de0 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.465+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5540043c0 con 0x7fd564101de0 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.465+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56419d5a0 con 0x7fd564101de0 2024-09-12T02:47:39.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.465+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554004a00 con 0x7fd564101de0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.465+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a4e80 con 0x7fd564101de0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.466+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5641021c0 con 0x7fd564101de0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.467+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd554004ba0 con 0x7fd564101de0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.468+0000 7fd561ffb640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 0x7fd5500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.468+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7fd5540cee70 con 0x7fd564101de0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.468+0000 7fd55bfff640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 0x7fd5500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.468+0000 7fd55bfff640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 0x7fd5500a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5640fdce0 tx=0x7fd54c005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:39.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.471+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd554014030 con 0x7fd564101de0 2024-09-12T02:47:39.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:39.649+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd564199660 con 0x7fd564101de0 2024-09-12T02:47:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.091+0000 7fd561ffb640 1 -- 172.21.15.46:0/2313601127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7fd55409c180 con 0x7fd564101de0 2024-09-12T02:47:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 msgr2=0x7fd5500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 0x7fd5500a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5640fdce0 tx=0x7fd54c005b60 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 msgr2=0x7fd564198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c890 tx=0x7fd55400cd50 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 shutdown_connections 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd5500a2c00 0x7fd5500a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd564106f10 0x7fd56419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd564102790 0x7fd564198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 --2- 172.21.15.46:0/2313601127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd564101de0 0x7fd564198300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 >> 172.21.15.46:0/2313601127 conn(0x7fd5640fd8c0 msgr2=0x7fd5640fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 shutdown_connections 2024-09-12T02:47:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:40.095+0000 7fd56a4bd640 1 -- 172.21.15.46:0/2313601127 wait complete. 2024-09-12T02:47:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:40 smithi145 ceph-mon[34674]: pgmap v279: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:40 smithi145 ceph-mon[34674]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:40.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:40 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T02:47:40.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:40 smithi046 ceph-mon[30951]: pgmap v279: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:40.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:40 smithi046 ceph-mon[30951]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:40.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:40 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T02:47:40.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:40 smithi196 ceph-mon[34587]: pgmap v279: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:40.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:40 smithi196 ceph-mon[34587]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:40.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:40 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T02:47:40.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T02:47:41.056 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:41.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:41 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T02:47:41.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:41 smithi145 ceph-mon[34674]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:47:41.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:41 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T02:47:41.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:41 smithi046 ceph-mon[30951]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:47:41.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:41 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2313601127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T02:47:41.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:41 smithi196 ceph-mon[34587]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 -- 172.21.15.46:0/1474803830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102570 msgr2=0x7fa35c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 --2- 172.21.15.46:0/1474803830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102570 0x7fa35c1029b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa344009900 tx=0x7fa34402a480 comp rx=0 tx=0).stop 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 -- 172.21.15.46:0/1474803830 shutdown_connections 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 --2- 172.21.15.46:0/1474803830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c102ef0 0x7fa35c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 --2- 172.21.15.46:0/1474803830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102570 0x7fa35c1029b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 --2- 172.21.15.46:0/1474803830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c100e20 0x7fa35c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 -- 172.21.15.46:0/1474803830 >> 172.21.15.46:0/1474803830 conn(0x7fa35c0fc7d0 msgr2=0x7fa35c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:41.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 -- 172.21.15.46:0/1474803830 shutdown_connections 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.578+0000 7fa362530640 1 -- 172.21.15.46:0/1474803830 wait complete. 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 Processor -- start 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 -- start start 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c100e20 0x7fa35c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c102570 0x7fa35c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c110e50 con 0x7fa35c102ef0 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c110cd0 con 0x7fa35c102570 2024-09-12T02:47:41.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa362530640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c110fd0 con 0x7fa35c100e20 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa360acd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa360acd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50438/0 (socket says 172.21.15.46:50438) 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa360acd640 1 -- 172.21.15.46:0/780808738 learned_addr learned my addr 172.21.15.46:0/780808738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa35bfff640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c100e20 0x7fa35c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.579+0000 7fa353fff640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c102570 0x7fa35c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 -- 172.21.15.46:0/780808738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c100e20 msgr2=0x7fa35c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c100e20 0x7fa35c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 -- 172.21.15.46:0/780808738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c102570 msgr2=0x7fa35c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c102570 0x7fa35c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 -- 172.21.15.46:0/780808738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c19d420 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa360acd640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b970 tx=0x7fa35400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354019070 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c19d710 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa354014440 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540075d0 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.580+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a4ff0 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.581+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c101200 con 0x7fa35c102ef0 2024-09-12T02:47:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.582+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa35401c070 con 0x7fa35c102ef0 2024-09-12T02:47:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.583+0000 7fa359ffb640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 0x7fa3480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.583+0000 7fa35bfff640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 0x7fa3480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.583+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fa3540ce910 con 0x7fa35c102ef0 2024-09-12T02:47:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.583+0000 7fa35bfff640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 0x7fa3480a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00bbe0 tx=0x7fa34c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.586+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa35409b3d0 con 0x7fa35c102ef0 2024-09-12T02:47:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:41.797+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa35c1997d0 con 0x7fa35c102ef0 2024-09-12T02:47:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.098+0000 7fa359ffb640 1 -- 172.21.15.46:0/780808738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa35409f6d0 con 0x7fa35c102ef0 2024-09-12T02:47:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T02:47:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 msgr2=0x7fa3480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 0x7fa3480a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00bbe0 tx=0x7fa34c00b390 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 msgr2=0x7fa35c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b970 tx=0x7fa35400be30 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 shutdown_connections 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa35c102ef0 0x7fa35c19cd40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa35c102570 0x7fa35c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fa3480a2b80 0x7fa3480a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 --2- 172.21.15.46:0/780808738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa35c100e20 0x7fa35c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 >> 172.21.15.46:0/780808738 conn(0x7fa35c0fc7d0 msgr2=0x7fa35c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 shutdown_connections 2024-09-12T02:47:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:42.102+0000 7fa362530640 1 -- 172.21.15.46:0/780808738 wait complete. 2024-09-12T02:47:42.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:42 smithi145 ceph-mon[34674]: pgmap v282: 97 pgs: 2 creating+activating, 7 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:42.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:42 smithi145 ceph-mon[34674]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:47:42.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:42 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:47:42.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:42 smithi145 ceph-mon[34674]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:47:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:42 smithi196 ceph-mon[34587]: pgmap v282: 97 pgs: 2 creating+activating, 7 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:42 smithi196 ceph-mon[34587]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:47:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:42 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:47:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:42 smithi196 ceph-mon[34587]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:47:42.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:42 smithi046 ceph-mon[30951]: pgmap v282: 97 pgs: 2 creating+activating, 7 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:42.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:42 smithi046 ceph-mon[30951]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:47:42.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:42 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:47:42.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:42 smithi046 ceph-mon[30951]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:47:42.851 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T02:47:42.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T02:47:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:43.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:43 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:47:43.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:43 smithi046 ceph-mon[30951]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:47:43.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:43 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:47:43.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:43 smithi046 ceph-mon[30951]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:47:43.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:43 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:47:43.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:43 smithi196 ceph-mon[34587]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:47:43.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:43 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:47:43.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:43 smithi196 ceph-mon[34587]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:47:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:43 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/780808738' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:47:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:43 smithi145 ceph-mon[34674]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:47:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:43 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:47:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:43 smithi145 ceph-mon[34674]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 -- 172.21.15.46:0/151698548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 msgr2=0x7f06a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 --2- 172.21.15.46:0/151698548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a8106e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0698009930 tx=0x7f069802a420 comp rx=0 tx=0).stop 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 -- 172.21.15.46:0/151698548 shutdown_connections 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 --2- 172.21.15.46:0/151698548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8101420 0x7f06a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 --2- 172.21.15.46:0/151698548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8100a80 0x7f06a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 --2- 172.21.15.46:0/151698548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a8106e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 -- 172.21.15.46:0/151698548 >> 172.21.15.46:0/151698548 conn(0x7f06a80fc7a0 msgr2=0x7f06a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 -- 172.21.15.46:0/151698548 shutdown_connections 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.614+0000 7f06aea16640 1 -- 172.21.15.46:0/151698548 wait complete. 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 Processor -- start 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 -- start start 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100a80 0x7f06a80fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8101420 0x7f06a80fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8110d70 con 0x7f06a8106a80 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8110bf0 con 0x7f06a8101420 2024-09-12T02:47:43.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06aea16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8110ef0 con 0x7f06a8100a80 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50456/0 (socket says 172.21.15.46:50456) 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 -- 172.21.15.46:0/3612079295 learned_addr learned my addr 172.21.15.46:0/3612079295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06a7fff640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100a80 0x7f06a80fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f069ffff640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8101420 0x7f06a80fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 -- 172.21.15.46:0/3612079295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100a80 msgr2=0x7f06a80fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100a80 0x7f06a80fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 -- 172.21.15.46:0/3612079295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8101420 msgr2=0x7f06a80fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8101420 0x7f06a80fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.615+0000 7f06acfb3640 1 -- 172.21.15.46:0/3612079295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81a6910 con 0x7f06a8106a80 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06acfb3640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06a000cad0 tx=0x7f06a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a000ae20 con 0x7f06a8106a80 2024-09-12T02:47:43.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06a0005020 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81a6c00 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a000f6b0 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.616+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a7190 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.617+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a8100ee0 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.618+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f06a00051d0 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.618+0000 7f06a5ffb640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 0x7f06940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.619+0000 7f06a7fff640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 0x7f06940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.619+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f06a00cef60 con 0x7f06a8106a80 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.619+0000 7f06a7fff640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 0x7f06940a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0698006010 tx=0x7f0698000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:43.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.623+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06a0014030 con 0x7f06a8106a80 2024-09-12T02:47:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:43.805+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f06a810a000 con 0x7f06a8106a80 2024-09-12T02:47:44.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.115+0000 7f06a5ffb640 1 -- 172.21.15.46:0/3612079295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f06a009bf70 con 0x7f06a8106a80 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 msgr2=0x7f06940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 0x7f06940a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0698006010 tx=0x7f0698000bc0 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 msgr2=0x7f06a80fd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06a000cad0 tx=0x7f06a000cf90 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 shutdown_connections 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a8106a80 0x7f06a80fd910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a8101420 0x7f06a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f06940a2c00 0x7f06940a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 --2- 172.21.15.46:0/3612079295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100a80 0x7f06a80fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 >> 172.21.15.46:0/3612079295 conn(0x7f06a80fc7a0 msgr2=0x7f06a806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 shutdown_connections 2024-09-12T02:47:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:44.118+0000 7f06aea16640 1 -- 172.21.15.46:0/3612079295 wait complete. 2024-09-12T02:47:44.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: pgmap v285: 97 pgs: 2 creating+activating, 15 creating+peering, 41 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T02:47:44.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:47:44.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: Cluster is now healthy 2024-09-12T02:47:44.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:active 2024-09-12T02:47:44.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:44 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T02:47:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: pgmap v285: 97 pgs: 2 creating+activating, 15 creating+peering, 41 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T02:47:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:47:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: Cluster is now healthy 2024-09-12T02:47:44.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:active 2024-09-12T02:47:44.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:44 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: pgmap v285: 97 pgs: 2 creating+activating, 15 creating+peering, 41 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: Cluster is now healthy 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] up:active 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:44.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:44 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T02:47:44.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T02:47:45.099 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:45.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:45 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T02:47:45.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:45 smithi046 ceph-mon[30951]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:47:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:45 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T02:47:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:45 smithi196 ceph-mon[34587]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:47:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:45 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3612079295' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T02:47:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:45 smithi145 ceph-mon[34674]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:47:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 -- 172.21.15.46:0/1019023159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 msgr2=0x7ff4c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1019023159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c4103120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ca0 tx=0x7ff4b402a950 comp rx=0 tx=0).stop 2024-09-12T02:47:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 -- 172.21.15.46:0/1019023159 shutdown_connections 2024-09-12T02:47:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1019023159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c410c170 0x7ff4c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1019023159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4c4103660 0x7ff4c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1019023159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c4103120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 -- 172.21.15.46:0/1019023159 >> 172.21.15.46:0/1019023159 conn(0x7ff4c40fc7a0 msgr2=0x7ff4c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 -- 172.21.15.46:0/1019023159 shutdown_connections 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.636+0000 7ff4ca051640 1 -- 172.21.15.46:0/1019023159 wait complete. 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 Processor -- start 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 -- start start 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 0x7ff4c419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4c410c170 0x7ff4c419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4111e70 con 0x7ff4c4102d40 2024-09-12T02:47:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4111cf0 con 0x7ff4c410c170 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4ca051640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4111ff0 con 0x7ff4c4103660 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53290/0 (socket says 172.21.15.46:53290) 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 -- 172.21.15.46:0/1722112374 learned_addr learned my addr 172.21.15.46:0/1722112374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c2ffd640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 0x7ff4c419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 -- 172.21.15.46:0/1722112374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 msgr2=0x7ff4c419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 0x7ff4c419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 -- 172.21.15.46:0/1722112374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4c410c170 msgr2=0x7ff4c419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4c410c170 0x7ff4c419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c37fe640 1 -- 172.21.15.46:0/1722112374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c419f4d0 con 0x7ff4c4102d40 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.637+0000 7ff4c2ffd640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 0x7ff4c419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:47:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4c37fe640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4005e30 tx=0x7ff4b4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4034040 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4b40308d0 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c419f760 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004830 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.638+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a70a0 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.639+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c4103120 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.640+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4b4037020 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.641+0000 7ff4c0ff9640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 0x7ff4ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.641+0000 7ff4c2ffd640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 0x7ff4ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.641+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7ff4b40f0060 con 0x7ff4c4102d40 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.641+0000 7ff4c2ffd640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 0x7ff4ac0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4102930 tx=0x7ff4b8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.645+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4b40bcf90 con 0x7ff4c4102d40 2024-09-12T02:47:45.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:45.825+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff4c41a72d0 con 0x7ff4c4102d40 2024-09-12T02:47:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:46 smithi145 ceph-mon[34674]: pgmap v287: 97 pgs: 1 creating+activating, 12 creating+peering, 67 active+clean, 17 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 493 B/s wr, 1 op/s 2024-09-12T02:47:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:46 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:47:46.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:46 smithi196 ceph-mon[34587]: pgmap v287: 97 pgs: 1 creating+activating, 12 creating+peering, 67 active+clean, 17 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 493 B/s wr, 1 op/s 2024-09-12T02:47:46.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:46 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:47:47.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:46 smithi046 ceph-mon[30951]: pgmap v287: 97 pgs: 1 creating+activating, 12 creating+peering, 67 active+clean, 17 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 493 B/s wr, 1 op/s 2024-09-12T02:47:47.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:46 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:47:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.129+0000 7ff4c0ff9640 1 -- 172.21.15.46:0/1722112374 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7ff4b40a5e20 con 0x7ff4c4102d40 2024-09-12T02:47:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-09-12T02:47:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.132+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 msgr2=0x7ff4ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.132+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 0x7ff4ac0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4102930 tx=0x7ff4b8005e30 comp rx=0 tx=0).stop 2024-09-12T02:47:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 msgr2=0x7ff4c419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4005e30 tx=0x7ff4b4003f90 comp rx=0 tx=0).stop 2024-09-12T02:47:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 shutdown_connections 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4c410c170 0x7ff4c419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff4ac0a2c00 0x7ff4ac0a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4c4103660 0x7ff4c419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 --2- 172.21.15.46:0/1722112374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c4102d40 0x7ff4c419a520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 >> 172.21.15.46:0/1722112374 conn(0x7ff4c40fc7a0 msgr2=0x7ff4c40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 shutdown_connections 2024-09-12T02:47:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:47.133+0000 7ff4ca051640 1 -- 172.21.15.46:0/1722112374 wait complete. 2024-09-12T02:47:47.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:47 smithi046 ceph-mon[30951]: pgmap v289: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T02:47:47.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:47 smithi046 ceph-mon[30951]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:47:47.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:47 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:47:47.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:47 smithi046 ceph-mon[30951]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:47:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:47 smithi145 ceph-mon[34674]: pgmap v289: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T02:47:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:47 smithi145 ceph-mon[34674]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:47:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:47 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:47:47.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:47 smithi145 ceph-mon[34674]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:47:47.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T02:47:47.970 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:47:47 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: 2024-09-12T02:47:47.600+0000 7f626d396640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T02:47:47.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:47 smithi196 ceph-mon[34587]: pgmap v289: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T02:47:47.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:47 smithi196 ceph-mon[34587]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:47:47.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:47 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1722112374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:47:47.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:47 smithi196 ceph-mon[34587]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:47:48.142 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 -- 172.21.15.46:0/1140226520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102ef0 msgr2=0x7f6ad410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 --2- 172.21.15.46:0/1140226520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102ef0 0x7f6ad410da40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009ca0 tx=0x7f6acc02a950 comp rx=0 tx=0).stop 2024-09-12T02:47:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 -- 172.21.15.46:0/1140226520 shutdown_connections 2024-09-12T02:47:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 --2- 172.21.15.46:0/1140226520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102ef0 0x7f6ad410da40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 --2- 172.21.15.46:0/1140226520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102570 0x7f6ad41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 --2- 172.21.15.46:0/1140226520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 -- 172.21.15.46:0/1140226520 >> 172.21.15.46:0/1140226520 conn(0x7f6ad40fc7d0 msgr2=0x7f6ad40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 -- 172.21.15.46:0/1140226520 shutdown_connections 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.642+0000 7f6ada412640 1 -- 172.21.15.46:0/1140226520 wait complete. 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 Processor -- start 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 -- start start 2024-09-12T02:47:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102ef0 0x7f6ad419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4110ea0 con 0x7f6ad4102570 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4110d20 con 0x7f6ad4102ef0 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ada412640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4111020 con 0x7f6ad4100e20 2024-09-12T02:47:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53304/0 (socket says 172.21.15.46:53304) 2024-09-12T02:47:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ad37fe640 1 -- 172.21.15.46:0/471315972 learned_addr learned my addr 172.21.15.46:0/471315972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.643+0000 7f6ad3fff640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad89af640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102ef0 0x7f6ad419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 -- 172.21.15.46:0/471315972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 msgr2=0x7f6ad4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 -- 172.21.15.46:0/471315972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102ef0 msgr2=0x7f6ad419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102ef0 0x7f6ad419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 -- 172.21.15.46:0/471315972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad419d3c0 con 0x7f6ad4102570 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad3fff640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:47:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad37fe640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800d950 tx=0x7f6ac800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8017070 con 0x7f6ad4102570 2024-09-12T02:47:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ac8009c30 con 0x7f6ad4102570 2024-09-12T02:47:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad419d6b0 con 0x7f6ad4102570 2024-09-12T02:47:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800f620 con 0x7f6ad4102570 2024-09-12T02:47:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.644+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a4f90 con 0x7f6ad4102570 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.645+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad4101200 con 0x7f6ad4102570 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.647+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6ac80109a0 con 0x7f6ad4102570 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.648+0000 7f6ad17fa640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 0x7f6abc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.648+0000 7f6ad3fff640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 0x7f6abc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.648+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f6ac80cef40 con 0x7f6ad4102570 2024-09-12T02:47:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.648+0000 7f6ad3fff640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 0x7f6abc0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009930 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.651+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ac8009de0 con 0x7f6ad4102570 2024-09-12T02:47:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:48.826+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6ad4199770 con 0x7f6ad4102570 2024-09-12T02:47:49.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.136+0000 7f6ad17fa640 1 -- 172.21.15.46:0/471315972 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7f6ad4199770 con 0x7f6ad4102570 2024-09-12T02:47:49.137 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T02:47:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.139+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 msgr2=0x7f6abc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.139+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 0x7f6abc0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009930 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-09-12T02:47:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.139+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 msgr2=0x7f6ad4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.139+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800d950 tx=0x7f6ac800de10 comp rx=0 tx=0).stop 2024-09-12T02:47:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 shutdown_connections 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ad4102ef0 0x7f6ad419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4102570 0x7f6ad4198950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6abc0a2c00 0x7f6abc0a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 --2- 172.21.15.46:0/471315972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad4100e20 0x7f6ad4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 >> 172.21.15.46:0/471315972 conn(0x7f6ad40fc7d0 msgr2=0x7f6ad406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 shutdown_connections 2024-09-12T02:47:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:49.140+0000 7f6ada412640 1 -- 172.21.15.46:0/471315972 wait complete. 2024-09-12T02:47:49.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:49 smithi196 ceph-mon[34587]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:47:49.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:49 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T02:47:49.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:49 smithi046 ceph-mon[30951]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:47:49.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:49 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T02:47:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:49 smithi145 ceph-mon[34674]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:47:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:49 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T02:47:49.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T02:47:50.121 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:50.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:50 smithi046 ceph-mon[30951]: pgmap v292: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-12T02:47:50.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:50 smithi046 ceph-mon[30951]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:50.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:50 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T02:47:50.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:50 smithi046 ceph-mon[30951]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:47:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:50 smithi196 ceph-mon[34587]: pgmap v292: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-12T02:47:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:50 smithi196 ceph-mon[34587]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:50 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T02:47:50.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:50 smithi196 ceph-mon[34587]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:47:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:50 smithi145 ceph-mon[34674]: pgmap v292: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 9 op/s 2024-09-12T02:47:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:50 smithi145 ceph-mon[34674]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:47:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:50 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/471315972' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T02:47:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:50 smithi145 ceph-mon[34674]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:47:50.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.632+0000 7f7488b54640 1 -- 172.21.15.46:0/1284397771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484102790 msgr2=0x7f74841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:50.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.632+0000 7f7488b54640 1 --2- 172.21.15.46:0/1284397771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484102790 0x7f74841067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f746c009900 tx=0x7f746c02a410 comp rx=0 tx=0).stop 2024-09-12T02:47:50.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.632+0000 7f7488b54640 1 -- 172.21.15.46:0/1284397771 shutdown_connections 2024-09-12T02:47:50.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.632+0000 7f7488b54640 1 --2- 172.21.15.46:0/1284397771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484106f10 0x7f748410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.632+0000 7f7488b54640 1 --2- 172.21.15.46:0/1284397771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484102790 0x7f74841067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 --2- 172.21.15.46:0/1284397771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f74841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 -- 172.21.15.46:0/1284397771 >> 172.21.15.46:0/1284397771 conn(0x7f74840fd8c0 msgr2=0x7f74840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 -- 172.21.15.46:0/1284397771 shutdown_connections 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 -- 172.21.15.46:0/1284397771 wait complete. 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 Processor -- start 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.633+0000 7f7488b54640 1 -- start start 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484102790 0x7f74841988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7484111e20 con 0x7f7484106f10 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7484111ca0 con 0x7f7484101de0 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7484111fa0 con 0x7f7484102790 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f748259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53320/0 (socket says 172.21.15.46:53320) 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 -- 172.21.15.46:0/2826564234 learned_addr learned my addr 172.21.15.46:0/2826564234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f748259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:48272/0 (socket says 172.21.15.46:48272) 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7481d9b640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484102790 0x7f74841988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 -- 172.21.15.46:0/2826564234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484102790 msgr2=0x7f74841988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484102790 0x7f74841988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 -- 172.21.15.46:0/2826564234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 msgr2=0x7f7484198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 -- 172.21.15.46:0/2826564234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748419d310 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f748259c640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7482d9d640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f747400eb10 tx=0x7f747400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474004b20 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748419d600 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7474004cc0 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.634+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474010690 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.635+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74841a4ee0 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.636+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74841021c0 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.637+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7474005020 con 0x7f7484106f10 2024-09-12T02:47:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.638+0000 7f747b7fe640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 0x7f74700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.638+0000 7f748259c640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 0x7f74700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.638+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f74740cf910 con 0x7f7484106f10 2024-09-12T02:47:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.639+0000 7f748259c640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 0x7f74700a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f747c009fc0 tx=0x7f747c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.642+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f747409c5f0 con 0x7f7484106f10 2024-09-12T02:47:50.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:50.855+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f748402d7c0 con 0x7f7484106f10 2024-09-12T02:47:51.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.143+0000 7f747b7fe640 1 -- 172.21.15.46:0/2826564234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f747409bef0 con 0x7f7484106f10 2024-09-12T02:47:51.144 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 msgr2=0x7f74700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 0x7f74700a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f747c009fc0 tx=0x7f747c005f90 comp rx=0 tx=0).stop 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 msgr2=0x7f748419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f747400eb10 tx=0x7f747400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 shutdown_connections 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7484106f10 0x7f748419cc30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f74700a2b80 0x7f74700a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484102790 0x7f74841988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 --2- 172.21.15.46:0/2826564234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7484101de0 0x7f7484198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.145+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 >> 172.21.15.46:0/2826564234 conn(0x7f74840fd8c0 msgr2=0x7f74840fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.146+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 shutdown_connections 2024-09-12T02:47:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:51.146+0000 7f7488b54640 1 -- 172.21.15.46:0/2826564234 wait complete. 2024-09-12T02:47:51.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:51 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T02:47:51.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:51 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T02:47:51.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:51 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T02:47:52.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:52.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:52 smithi046 ceph-mon[30951]: pgmap v294: 129 pgs: 7 creating+peering, 13 unknown, 109 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s wr, 3 op/s 2024-09-12T02:47:52.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:52 smithi046 ceph-mon[30951]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:47:52.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:52 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T02:47:52.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:52 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:52 smithi196 ceph-mon[34587]: pgmap v294: 129 pgs: 7 creating+peering, 13 unknown, 109 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s wr, 3 op/s 2024-09-12T02:47:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:52 smithi196 ceph-mon[34587]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:47:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:52 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T02:47:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:52 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:52 smithi145 ceph-mon[34674]: pgmap v294: 129 pgs: 7 creating+peering, 13 unknown, 109 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s wr, 3 op/s 2024-09-12T02:47:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:52 smithi145 ceph-mon[34674]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:47:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:52 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2826564234' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T02:47:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:52 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:47:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/749952062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 msgr2=0x7fcc7810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/749952062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc7810bb50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009900 tx=0x7fcc6c02a480 comp rx=0 tx=0).stop 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/749952062 shutdown_connections 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/749952062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7810c170 0x7fcc7810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/749952062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc7810bb50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/749952062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc78102d40 0x7fcc78103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/749952062 >> 172.21.15.46:0/749952062 conn(0x7fcc780fc7a0 msgr2=0x7fcc780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/749952062 shutdown_connections 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.798+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/749952062 wait complete. 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 Processor -- start 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 -- start start 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc78102d40 0x7fcc781009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc7810c170 0x7fcc78101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78111e20 con 0x7fcc78103660 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78111ca0 con 0x7fcc78102d40 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc7d6a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78111fa0 con 0x7fcc7810c170 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc767fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc767fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53324/0 (socket says 172.21.15.46:53324) 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc767fc640 1 -- 172.21.15.46:0/1428072639 learned_addr learned my addr 172.21.15.46:0/1428072639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc76ffd640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc78102d40 0x7fcc781009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.799+0000 7fcc777fe640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc7810c170 0x7fcc78101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 -- 172.21.15.46:0/1428072639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc7810c170 msgr2=0x7fcc78101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc7810c170 0x7fcc78101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 -- 172.21.15.46:0/1428072639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc78102d40 msgr2=0x7fcc781009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc78102d40 0x7fcc781009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 -- 172.21.15.46:0/1428072639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc781a68a0 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc767fc640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005940 tx=0x7fcc6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c03f030 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc781a6b30 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc6c003cb0 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c035070 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.800+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc781a7010 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.801+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc78103120 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.802+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc6c042030 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.803+0000 7fcc5ffff640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 0x7fcc600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.803+0000 7fcc76ffd640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 0x7fcc600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.804+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fcc6c0efc10 con 0x7fcc78103660 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.804+0000 7fcc76ffd640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 0x7fcc600a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005fd0 tx=0x7fcc6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:52.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:52.807+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc6c0bc280 con 0x7fcc78103660 2024-09-12T02:47:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.021+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc7802d7c0 con 0x7fcc78103660 2024-09-12T02:47:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.021+0000 7fcc5ffff640 1 -- 172.21.15.46:0/1428072639 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7fcc6c0c0580 con 0x7fcc78103660 2024-09-12T02:47:53.128 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:53.128 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-12T02:47:51:138776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:47:50.856371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24478,"qdb_cluster":[24478]},"id":1}]} 2024-09-12T02:47:53.129 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-12T02:47:53.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 msgr2=0x7fcc600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 0x7fcc600a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005fd0 tx=0x7fcc6800b390 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 msgr2=0x7fcc78100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005940 tx=0x7fcc6c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 shutdown_connections 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc600a2b80 0x7fcc600a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc7810c170 0x7fcc78101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc78103660 0x7fcc78100ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 --2- 172.21.15.46:0/1428072639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc78102d40 0x7fcc781009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 >> 172.21.15.46:0/1428072639 conn(0x7fcc780fc7a0 msgr2=0x7fcc780fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 shutdown_connections 2024-09-12T02:47:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:53.024+0000 7fcc7d6a0640 1 -- 172.21.15.46:0/1428072639 wait complete. 2024-09-12T02:47:53.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:53 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1428072639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:53.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:53 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1428072639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:53.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:53 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1428072639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:53.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:47:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:54.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:54 smithi046 ceph-mon[30951]: pgmap v296: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:54.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:54 smithi046 ceph-mon[30951]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:47:54.393 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:54 smithi046 ceph-mon[30951]: Cluster is now healthy 2024-09-12T02:47:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:54 smithi196 ceph-mon[34587]: pgmap v296: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:54 smithi196 ceph-mon[34587]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:47:54.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:54 smithi196 ceph-mon[34587]: Cluster is now healthy 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 -- 172.21.15.46:0/950470470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4102da0 msgr2=0x7fddb4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 --2- 172.21.15.46:0/950470470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4102da0 0x7fddb4103180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009900 tx=0x7fdda402a480 comp rx=0 tx=0).stop 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 -- 172.21.15.46:0/950470470 shutdown_connections 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 --2- 172.21.15.46:0/950470470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 0x7fddb410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 --2- 172.21.15.46:0/950470470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb41036c0 0x7fddb410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 --2- 172.21.15.46:0/950470470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb4102da0 0x7fddb4103180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 -- 172.21.15.46:0/950470470 >> 172.21.15.46:0/950470470 conn(0x7fddb40fc7a0 msgr2=0x7fddb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 -- 172.21.15.46:0/950470470 shutdown_connections 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.473+0000 7fddb8dde640 1 -- 172.21.15.46:0/950470470 wait complete. 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 Processor -- start 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 -- start start 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb4102da0 0x7fddb4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 0x7fddb4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111e10 con 0x7fddb41036c0 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111c90 con 0x7fddb4102da0 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.474+0000 7fddb8dde640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111f90 con 0x7fddb410c180 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52486/0 (socket says 172.21.15.46:52486) 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb4102da0 0x7fddb4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 -- 172.21.15.46:0/2598463853 learned_addr learned my addr 172.21.15.46:0/2598463853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb2d9d640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 0x7fddb4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 -- 172.21.15.46:0/2598463853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 msgr2=0x7fddb4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 0x7fddb4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 -- 172.21.15.46:0/2598463853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb4102da0 msgr2=0x7fddb4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb4102da0 0x7fddb4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 -- 172.21.15.46:0/2598463853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41a6830 con 0x7fddb41036c0 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb1d9b640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdda800eab0 tx=0x7fdda800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda800ce40 con 0x7fddb41036c0 2024-09-12T02:47:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb41a6b20 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdda8011920 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8010a00 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.475+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb41a7060 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.476+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb4103180 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.478+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdda8011ad0 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.478+0000 7fdda37fe640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 0x7fdd9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.478+0000 7fddb259c640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 0x7fdd9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.478+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fdda80cf820 con 0x7fddb41036c0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.479+0000 7fddb259c640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 0x7fdd9c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdda4031440 tx=0x7fdda4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.483+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdda8016030 con 0x7fddb41036c0 2024-09-12T02:47:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:54 smithi145 ceph-mon[34674]: pgmap v296: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:54 smithi145 ceph-mon[34674]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:47:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:54 smithi145 ceph-mon[34674]: Cluster is now healthy 2024-09-12T02:47:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.660+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddb4106c90 con 0x7fddb41036c0 2024-09-12T02:47:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.662+0000 7fdda37fe640 1 -- 172.21.15.46:0/2598463853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7fdda809c540 con 0x7fddb41036c0 2024-09-12T02:47:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:54.663 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.664+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 msgr2=0x7fdd9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.664+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 0x7fdd9c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdda4031440 tx=0x7fdda4005b50 comp rx=0 tx=0).stop 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.664+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 msgr2=0x7fddb4100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.664+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdda800eab0 tx=0x7fdda800ef70 comp rx=0 tx=0).stop 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 shutdown_connections 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdd9c0a2b80 0x7fdd9c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb410c180 0x7fddb4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb41036c0 0x7fddb4100f40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 --2- 172.21.15.46:0/2598463853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddb4102da0 0x7fddb4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 >> 172.21.15.46:0/2598463853 conn(0x7fddb40fc7a0 msgr2=0x7fddb40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 shutdown_connections 2024-09-12T02:47:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:54.665+0000 7fddb8dde640 1 -- 172.21.15.46:0/2598463853 wait complete. 2024-09-12T02:47:55.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:55 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2598463853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:55.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:55 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2598463853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:55.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:47:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:55 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2598463853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:55.799 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:56.210 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:56 smithi046 ceph-mon[30951]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3122025555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8101420 msgr2=0x7f10c810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3122025555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8101420 0x7f10c810d950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009cd0 tx=0x7f10c002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3122025555 shutdown_connections 2024-09-12T02:47:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3122025555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8101420 0x7f10c810d950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3122025555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8100a80 0x7f10c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3122025555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3122025555 >> 172.21.15.46:0/3122025555 conn(0x7f10c80fc7a0 msgr2=0x7f10c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3122025555 shutdown_connections 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3122025555 wait complete. 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.292+0000 7f10cf5f0640 1 Processor -- start 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 -- start start 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8101420 0x7f10c81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8110d30 con 0x7f10c8100a80 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8110bb0 con 0x7f10c8106a80 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cf5f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8110eb0 con 0x7f10c8101420 2024-09-12T02:47:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52502/0 (socket says 172.21.15.46:52502) 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 -- 172.21.15.46:0/3931718745 learned_addr learned my addr 172.21.15.46:0/3931718745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10ccb8b640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8101420 0x7f10c81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cdb8d640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 -- 172.21.15.46:0/3931718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8101420 msgr2=0x7f10c81976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8101420 0x7f10c81976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 -- 172.21.15.46:0/3931718745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 msgr2=0x7f10c819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c819ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 -- 172.21.15.46:0/3931718745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c819c160 con 0x7f10c8100a80 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cdb8d640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.293+0000 7f10cd38c640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c920 tx=0x7f10c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.294+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4007c70 con 0x7f10c8100a80 2024-09-12T02:47:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.294+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c819c450 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.294+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10c4007e10 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.294+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4018820 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.294+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c81a3d30 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.295+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8100ee0 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.296+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f10c40189c0 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.297+0000 7f10be7fc640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 0x7f10b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.297+0000 7f10ccb8b640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 0x7f10b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.297+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f10c40cf380 con 0x7f10c8100a80 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.297+0000 7f10ccb8b640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 0x7f10b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10c80fed60 tx=0x7f10b800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.301+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10c4014030 con 0x7f10c8100a80 2024-09-12T02:47:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:56 smithi196 ceph-mon[34587]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.478+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10c81a3fc0 con 0x7f10c8100a80 2024-09-12T02:47:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.480+0000 7f10be7fc640 1 -- 172.21.15.46:0/3931718745 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7f10c409c0a0 con 0x7f10c8100a80 2024-09-12T02:47:56.480 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:56.481 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:47:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 msgr2=0x7f10b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 0x7f10b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10c80fed60 tx=0x7f10b800a250 comp rx=0 tx=0).stop 2024-09-12T02:47:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 msgr2=0x7f10c81971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c920 tx=0x7f10c400cde0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 shutdown_connections 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10c8106a80 0x7f10c819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f10b40a2b80 0x7f10b40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c8101420 0x7f10c81976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 --2- 172.21.15.46:0/3931718745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c8100a80 0x7f10c81971b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 >> 172.21.15.46:0/3931718745 conn(0x7f10c80fc7a0 msgr2=0x7f10c80fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 shutdown_connections 2024-09-12T02:47:56.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:56.482+0000 7f10cf5f0640 1 -- 172.21.15.46:0/3931718745 wait complete. 2024-09-12T02:47:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:56 smithi145 ceph-mon[34674]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:57.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:47:57.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:57 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3931718745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:57.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:57 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3931718745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:57.606 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:57.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:57 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3931718745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 -- 172.21.15.46:0/3355238411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64101420 msgr2=0x7fca6410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/3355238411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64101420 0x7fca6410d950 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009cd0 tx=0x7fca5c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 -- 172.21.15.46:0/3355238411 shutdown_connections 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/3355238411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64101420 0x7fca6410d950 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/3355238411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 0x7fca64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/3355238411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64106a80 0x7fca64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 -- 172.21.15.46:0/3355238411 >> 172.21.15.46:0/3355238411 conn(0x7fca640fc7a0 msgr2=0x7fca640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 -- 172.21.15.46:0/3355238411 shutdown_connections 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.087+0000 7fca6c2bc640 1 -- 172.21.15.46:0/3355238411 wait complete. 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 Processor -- start 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 -- start start 2024-09-12T02:47:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 0x7fca640fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 0x7fca640fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca64110d50 con 0x7fca64106a80 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fca64110bd0 con 0x7fca64100a80 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6c2bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca64110ed0 con 0x7fca64101420 2024-09-12T02:47:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52514/0 (socket says 172.21.15.46:52514) 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 -- 172.21.15.46:0/538431039 learned_addr learned my addr 172.21.15.46:0/538431039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca69857640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 0x7fca640fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a058640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 0x7fca640fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 -- 172.21.15.46:0/538431039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 msgr2=0x7fca640fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 0x7fca640fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 -- 172.21.15.46:0/538431039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 msgr2=0x7fca640fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 0x7fca640fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca6a859640 1 -- 172.21.15.46:0/538431039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a6840 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.088+0000 7fca69857640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 0x7fca640fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca6a859640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005cc0 tx=0x7fca5c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c004450 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a6ad0 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca5c031c90 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c03b400 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.089+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a7060 con 0x7fca64106a80 2024-09-12T02:47:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.090+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca64100ee0 con 0x7fca64106a80 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.091+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca5c004680 con 0x7fca64106a80 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.092+0000 7fca5b7fe640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 0x7fca500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.092+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fca5c02e030 con 0x7fca64106a80 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.092+0000 7fca6a058640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 0x7fca500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.093+0000 7fca6a058640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 0x7fca500a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca60005fd0 tx=0x7fca60005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.096+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca5c0bd1d0 con 0x7fca64106a80 2024-09-12T02:47:58.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:58 smithi046 ceph-mon[30951]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.311+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca6410a020 con 0x7fca64106a80 2024-09-12T02:47:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.312+0000 7fca5b7fe640 1 -- 172.21.15.46:0/538431039 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7fca5c0098b0 con 0x7fca64106a80 2024-09-12T02:47:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:58.316 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-12T02:47:51:138776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:47:50.856371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24478,"qdb_cluster":[24478]},"id":1}]} 2024-09-12T02:47:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-12T02:47:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 msgr2=0x7fca500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 0x7fca500a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca60005fd0 tx=0x7fca60005850 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 msgr2=0x7fca640fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005cc0 tx=0x7fca5c003c20 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 shutdown_connections 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca64106a80 0x7fca640fd8f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca500a2b80 0x7fca500a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64101420 0x7fca640fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 --2- 172.21.15.46:0/538431039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca64100a80 0x7fca640fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 >> 172.21.15.46:0/538431039 conn(0x7fca640fc7a0 msgr2=0x7fca6406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 shutdown_connections 2024-09-12T02:47:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:58.314+0000 7fca6c2bc640 1 -- 172.21.15.46:0/538431039 wait complete. 2024-09-12T02:47:58.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:58 smithi196 ceph-mon[34587]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:58 smithi145 ceph-mon[34674]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:47:59.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:47:59.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:47:59 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/538431039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:47:59.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:47:59 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/538431039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:59.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:47:59 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/538431039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:47:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 -- 172.21.15.46:0/2827515254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354106f10 msgr2=0x7f435410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 --2- 172.21.15.46:0/2827515254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354106f10 0x7f435410ea90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f434c009ca0 tx=0x7f434c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 -- 172.21.15.46:0/2827515254 shutdown_connections 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 --2- 172.21.15.46:0/2827515254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354106f10 0x7f435410ea90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 --2- 172.21.15.46:0/2827515254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354102790 0x7f43541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 --2- 172.21.15.46:0/2827515254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354101de0 0x7f43541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 -- 172.21.15.46:0/2827515254 >> 172.21.15.46:0/2827515254 conn(0x7f43540fd8c0 msgr2=0x7f43540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 -- 172.21.15.46:0/2827515254 shutdown_connections 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.795+0000 7f4358fab640 1 -- 172.21.15.46:0/2827515254 wait complete. 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 Processor -- start 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 -- start start 2024-09-12T02:47:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354102790 0x7f4354198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 0x7f435419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111e40 con 0x7f4354101de0 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111cc0 con 0x7f4354102790 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4358fab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111fc0 con 0x7f4354106f10 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f435259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f435259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52534/0 (socket says 172.21.15.46:52534) 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f435259c640 1 -- 172.21.15.46:0/3098340765 learned_addr learned my addr 172.21.15.46:0/3098340765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4351d9b640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354102790 0x7f4354198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.796+0000 7f4352d9d640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 0x7f435419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 -- 172.21.15.46:0/3098340765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 msgr2=0x7f435419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 0x7f435419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 -- 172.21.15.46:0/3098340765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354102790 msgr2=0x7f4354198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354102790 0x7f4354198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 -- 172.21.15.46:0/3098340765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435419d2d0 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f4352d9d640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 0x7f435419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f435259c640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f434400c920 tx=0x7f434400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344007bb0 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4344007d50 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435419d5c0 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344018730 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.797+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541a4ea0 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.798+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43541021c0 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.799+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f434400f450 con 0x7f4354101de0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.800+0000 7f43437fe640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 0x7f433c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.800+0000 7f4351d9b640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 0x7f433c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:47:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.800+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f43440cf3b0 con 0x7f4354101de0 2024-09-12T02:47:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.800+0000 7f4351d9b640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 0x7f433c0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43541088e0 tx=0x7f4348009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:47:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.804+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4344014030 con 0x7f4354101de0 2024-09-12T02:47:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.980+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4354199680 con 0x7f4354101de0 2024-09-12T02:47:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.982+0000 7f43437fe640 1 -- 172.21.15.46:0/3098340765 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7f434409c0d0 con 0x7f4354101de0 2024-09-12T02:47:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:47:59.983 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 msgr2=0x7f433c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 0x7f433c0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43541088e0 tx=0x7f4348009110 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 msgr2=0x7f4354198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f434400c920 tx=0x7f434400cd20 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 shutdown_connections 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f433c0a2b80 0x7f433c0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4354106f10 0x7f435419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4354102790 0x7f4354198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 --2- 172.21.15.46:0/3098340765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4354101de0 0x7f4354198320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:47:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 >> 172.21.15.46:0/3098340765 conn(0x7f43540fd8c0 msgr2=0x7f43540fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:47:59.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 shutdown_connections 2024-09-12T02:47:59.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:47:59.984+0000 7f4358fab640 1 -- 172.21.15.46:0/3098340765 wait complete. 2024-09-12T02:48:00.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:00 smithi046 ceph-mon[30951]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:00.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:00 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3098340765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:00 smithi196 ceph-mon[34587]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:00 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3098340765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:00.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:00 smithi145 ceph-mon[34674]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:00.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:00 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3098340765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:00.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T02:47:41.799879+0000', 'modified': '2024-09-12T02:47:50.856371+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6839/1262634008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1262634008}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1262634008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24478, 'qdb_cluster': [24478]} 2024-09-12T02:48:00.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T02:48:00.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:48:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- 172.21.15.46:0/2339779580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260101420 msgr2=0x7f426010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 --2- 172.21.15.46:0/2339779580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260101420 0x7f426010d950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4258009cd0 tx=0x7f425802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- 172.21.15.46:0/2339779580 shutdown_connections 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 --2- 172.21.15.46:0/2339779580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260101420 0x7f426010d950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 --2- 172.21.15.46:0/2339779580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f4260100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 --2- 172.21.15.46:0/2339779580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260106a80 0x7f4260106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- 172.21.15.46:0/2339779580 >> 172.21.15.46:0/2339779580 conn(0x7f42600fc7a0 msgr2=0x7f42600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- 172.21.15.46:0/2339779580 shutdown_connections 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- 172.21.15.46:0/2339779580 wait complete. 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 Processor -- start 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.505+0000 7f4267b78640 1 -- start start 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f42601991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 0x7f4260199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4260110d50 con 0x7f4260106a80 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4260110bd0 con 0x7f4260101420 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4267b78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4260110ed0 con 0x7f4260100a80 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52546/0 (socket says 172.21.15.46:52546) 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 -- 172.21.15.46:0/2409524481 learned_addr learned my addr 172.21.15.46:0/2409524481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4265914640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f42601991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4265113640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 0x7f4260199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 -- 172.21.15.46:0/2409524481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 msgr2=0x7f42601991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f42601991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 -- 172.21.15.46:0/2409524481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 msgr2=0x7f4260199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 0x7f4260199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 -- 172.21.15.46:0/2409524481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42601a4780 con 0x7f4260106a80 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4265113640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 0x7f4260199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4265914640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f42601991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4266115640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4258005cc0 tx=0x7f4258003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.506+0000 7f4256ffd640 1 -- 172.21.15.46:0/2409524481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258004240 con 0x7f4260106a80 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.507+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42601a4950 con 0x7f4260106a80 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.507+0000 7f4256ffd640 1 -- 172.21.15.46:0/2409524481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42580305a0 con 0x7f4260106a80 2024-09-12T02:48:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.507+0000 7f4256ffd640 1 -- 172.21.15.46:0/2409524481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580466c0 con 0x7f4260106a80 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.507+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42601a4de0 con 0x7f4260106a80 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.509+0000 7f4256ffd640 1 -- 172.21.15.46:0/2409524481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4258030ac0 con 0x7f4260106a80 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.509+0000 7f4269f51740 1 Processor -- start 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.509+0000 7f4269f51740 1 -- start start 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcd761a090 con 0x55bcd76c2fc0 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcd75c1a70 con 0x55bcd76b41d0 2024-09-12T02:48:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4269f51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bcd7616050 con 0x55bcd76b38b0 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52554/0 (socket says 172.21.15.46:52554) 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 -- 172.21.15.46:0/1175991071 learned_addr learned my addr 172.21.15.46:0/1175991071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4256ffd640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 0x7f424c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265914640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4266115640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 0x7f424c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 -- 172.21.15.46:0/1175991071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 msgr2=0x55bcd76b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 -- 172.21.15.46:0/1175991071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 msgr2=0x55bcd76c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265113640 1 -- 172.21.15.46:0/1175991071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd75ce130 con 0x55bcd76c2fc0 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4265914640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.510+0000 7f4256ffd640 1 -- 172.21.15.46:0/2409524481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f425802e030 con 0x7f4260106a80 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.511+0000 7f4265113640 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4250016560 tx=0x7f425000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef22b179e119496 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.511+0000 7f424bfff640 1 -- 172.21.15.46:0/1175991071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250018430 con 0x55bcd76c2fc0 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.511+0000 7f4266115640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 0x7f424c0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f425804c000 tx=0x7f425803d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.511+0000 7f424bfff640 1 -- 172.21.15.46:0/1175991071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4250018a80 con 0x55bcd76c2fc0 2024-09-12T02:48:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.511+0000 7f424bfff640 1 -- 172.21.15.46:0/1175991071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500177d0 con 0x55bcd76c2fc0 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- 172.21.15.46:0/1175991071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 msgr2=0x55bcd76c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4250016560 tx=0x7f425000a340 comp rx=0 tx=0).stop 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- 172.21.15.46:0/1175991071 shutdown_connections 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c53b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 --2- 172.21.15.46:0/1175991071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- 172.21.15.46:0/1175991071 >> 172.21.15.46:0/1175991071 conn(0x55bcd76aff60 msgr2=0x55bcd76b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- 172.21.15.46:0/1175991071 shutdown_connections 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- 172.21.15.46:0/1175991071 wait complete. 2024-09-12T02:48:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 -- start start 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.512+0000 7f4269f51740 1 Processor -- start 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76c5b90 con 0x55bcd76c2fc0 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcd75d4f20 con 0x55bcd76b41d0 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76c5d10 con 0x55bcd76b38b0 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4269f51740 10 client.0 _ll_register_callbacks cb 0x55bcd759e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52566/0 (socket says 172.21.15.46:52566) 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 -- 172.21.15.46:0/1744186093 learned_addr learned my addr 172.21.15.46:0/1744186093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265914640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 -- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 msgr2=0x55bcd76c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 -- 172.21.15.46:0/1744186093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 msgr2=0x55bcd76bfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76bfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 -- 172.21.15.46:0/1744186093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd738b2b0 con 0x55bcd76c2fc0 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265914640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f4265113640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f425001a040 tx=0x7f42500174f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6c6e909fef9974 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f42497fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:48:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.513+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425002b060 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4250018a80 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f4250018840 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4250032050 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 10 client.14679 fetch_fsmap learned FSMap version 17 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55bcd76d1520 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4250018a80 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f4250024040 con 0x55bcd76c2fc0 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 10 client.14679 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 10 client.14679 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 20 client.14679 populate_metadata read hostname 'smithi046' 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd76d3740 0x55bcd76d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 4 client.14679 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd7627600 con 0x55bcd76d3740 2024-09-12T02:48:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.514+0000 7f4265914640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd76d3740 0x55bcd76d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.515+0000 7f4265914640 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd76d3740 0x55bcd76d4bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35af676a46a01ea7 server_cookie=ce36f03338ceebc1 in_seq=0 out_seq=0 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.515+0000 7f42497fa640 10 client.14679 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f42497fa640 1 -- 172.21.15.46:0/1744186093 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcd7627600 con 0x55bcd76d3740 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f42497fa640 10 client.14679 handle_command_reply: tid=0 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 client.14679 shutdown 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 msgr2=0x55bcd76c00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f425001a040 tx=0x7f42500174f0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 shutdown_connections 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd76d3740 0x55bcd76d4bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76c00f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 --2- 172.21.15.46:0/1744186093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b38b0 0x55bcd76c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f42497fa640 0 client.14679 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:48:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 >> 172.21.15.46:0/1744186093 conn(0x55bcd76aff60 msgr2=0x55bcd76b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 shutdown_connections 2024-09-12T02:48:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 1 -- 172.21.15.46:0/1744186093 wait complete. 2024-09-12T02:48:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.516+0000 7f4269f51740 20 client.14679 trim_cache size 0 max 16384 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.534+0000 7f4269f51740 1 Processor -- start 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.534+0000 7f4269f51740 1 -- start start 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76c2fc0 0x55bcd76bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d2ef0 con 0x55bcd76b38b0 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d35c0 con 0x55bcd76b41d0 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4269f51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d3c90 con 0x55bcd76c2fc0 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52578/0 (socket says 172.21.15.46:52578) 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 -- 172.21.15.46:0/1079207648 learned_addr learned my addr 172.21.15.46:0/1079207648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 -- 172.21.15.46:0/1079207648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76c2fc0 msgr2=0x55bcd76bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76c2fc0 0x55bcd76bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 -- 172.21.15.46:0/1079207648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 msgr2=0x55bcd76af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265914640 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 -- 172.21.15.46:0/1079207648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd76b4000 con 0x55bcd76b38b0 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f4265113640 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f425000a4a0 tx=0x7f4250016460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa398bc7272bf852 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f424b7fe640 1 -- 172.21.15.46:0/1079207648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425003a070 con 0x55bcd76b38b0 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.535+0000 7f424b7fe640 1 -- 172.21.15.46:0/1079207648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425000a730 con 0x55bcd76b38b0 2024-09-12T02:48:01.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f424b7fe640 1 -- 172.21.15.46:0/1079207648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425002b070 con 0x55bcd76b38b0 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 -- 172.21.15.46:0/1079207648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 msgr2=0x55bcd76d4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f425000a4a0 tx=0x7f4250016460 comp rx=0 tx=0).stop 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 -- 172.21.15.46:0/1079207648 shutdown_connections 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76c2fc0 0x55bcd76bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b41d0 0x55bcd76af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 --2- 172.21.15.46:0/1079207648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76b38b0 0x55bcd76d4bb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 -- 172.21.15.46:0/1079207648 >> 172.21.15.46:0/1079207648 conn(0x55bcd76aff60 msgr2=0x55bcd76b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 -- 172.21.15.46:0/1079207648 shutdown_connections 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.536+0000 7f4269f51740 1 -- 172.21.15.46:0/1079207648 wait complete. 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 -- start start 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 Processor -- start 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b38b0 0x55bcd76d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 0x55bcd76d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d4a30 con 0x55bcd76c2fc0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d4360 con 0x55bcd76b38b0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bcd76d5100 con 0x55bcd76b41d0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4269f51740 10 client.0 _ll_register_callbacks cb 0x7f425000acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4265113640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52594/0 (socket says 172.21.15.46:52594) 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4265113640 1 -- 172.21.15.46:0/3268059956 learned_addr learned my addr 172.21.15.46:0/3268059956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.537+0000 7f4265914640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 0x55bcd76d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 -- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 msgr2=0x55bcd76d6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 0x55bcd76d6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 -- 172.21.15.46:0/3268059956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b38b0 msgr2=0x55bcd76d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b38b0 0x55bcd76d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 -- 172.21.15.46:0/3268059956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd773c0e0 con 0x55bcd76c2fc0 2024-09-12T02:48:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265914640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 0x55bcd76d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4265113640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4250023660 tx=0x7f4250006ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e07c8ce2a0424d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425002b030 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f42500059c0 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425000a730 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f425004a050 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4269f51740 10 client.14685 fetch_fsmap learned FSMap version 17 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55bcd773d3e0 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425000a730 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.538+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f4250012050 con 0x55bcd76c2fc0 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 10 client.14685 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 10 client.14685 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 20 client.14685 populate_metadata read hostname 'smithi046' 2024-09-12T02:48:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd7741b70 0x55bcd7740880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 4 client.14685 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "status"}) -- 0x55bcd7621b10 con 0x55bcd7741b70 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4265914640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd7741b70 0x55bcd7740880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f4265914640 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd7741b70 0x55bcd7740880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2de620220992a4 server_cookie=46334cb80b17a3cb in_seq=0 out_seq=0 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.539+0000 7f42497fa640 10 client.14685 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f42497fa640 1 -- 172.21.15.46:0/3268059956 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55bcd7621b10 con 0x55bcd7741b70 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f42497fa640 10 client.14685 handle_command_reply: tid=0 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 client.14685 shutdown 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 msgr2=0x55bcd76d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4250023660 tx=0x7f4250006ab0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 shutdown_connections 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bcd7741b70 0x55bcd7740880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcd76c2fc0 0x55bcd76d6880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bcd76b41d0 0x55bcd76d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f42497fa640 0 client.14685 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 --2- 172.21.15.46:0/3268059956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcd76b38b0 0x55bcd76d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 >> 172.21.15.46:0/3268059956 conn(0x55bcd76aff60 msgr2=0x55bcd76bb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 shutdown_connections 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 1 -- 172.21.15.46:0/3268059956 wait complete. 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.540+0000 7f4269f51740 20 client.14685 trim_cache size 0 max 16384 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:48:01.547 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24478, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 19.438316577999998, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 88, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 39.685829708 2024-09-12T02:48:01.548 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 msgr2=0x7f424c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 0x7f424c0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f425804c000 tx=0x7f425803d040 comp rx=0 tx=0).stop 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 msgr2=0x7f42601a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4258005cc0 tx=0x7f4258003ba0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 shutdown_connections 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4260106a80 0x7f42601a4240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260101420 0x7f4260199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f424c0a2b80 0x7f424c0a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 --2- 172.21.15.46:0/2409524481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4260100a80 0x7f42601991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 >> 172.21.15.46:0/2409524481 conn(0x7f42600fc7a0 msgr2=0x7f42600fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 shutdown_connections 2024-09-12T02:48:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:01.541+0000 7f4267b78640 1 -- 172.21.15.46:0/2409524481 wait complete. 2024-09-12T02:48:02.332 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T02:48:02.332 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:02.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T02:48:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:02 smithi196 ceph-mon[34587]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:02 smithi046 ceph-mon[30951]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:02.595 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:02 smithi145 ceph-mon[34674]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:03.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2392460608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 msgr2=0x7f48f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2392460608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f010ea90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2392460608 shutdown_connections 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2392460608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f010ea90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2392460608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2392460608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 0x7f48f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2392460608 >> 172.21.15.46:0/2392460608 conn(0x7f48f00fd8c0 msgr2=0x7f48f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:03.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2392460608 shutdown_connections 2024-09-12T02:48:03.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.318+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2392460608 wait complete. 2024-09-12T02:48:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 Processor -- start 2024-09-12T02:48:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 -- start start 2024-09-12T02:48:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 0x7f48f0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0111ec0 con 0x7f48f0106f10 2024-09-12T02:48:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0111d40 con 0x7f48f0101de0 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f7fd1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0112040 con 0x7f48f0102790 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52620/0 (socket says 172.21.15.46:52620) 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 -- 172.21.15.46:0/2700520470 learned_addr learned my addr 172.21.15.46:0/2700520470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f556c640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f5d6d640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 0x7f48f0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 -- 172.21.15.46:0/2700520470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 msgr2=0x7f48f01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 -- 172.21.15.46:0/2700520470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 msgr2=0x7f48f0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.319+0000 7f48f656e640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 0x7f48f0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48f656e640 1 -- 172.21.15.46:0/2700520470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f019d430 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48f556c640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48f656e640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009dc0 tx=0x7f48e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004040 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f019d6c0 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48e8004690 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8030d90 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.320+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a5000 con 0x7f48f0106f10 2024-09-12T02:48:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.321+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f01021c0 con 0x7f48f0106f10 2024-09-12T02:48:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.322+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48e80305c0 con 0x7f48f0106f10 2024-09-12T02:48:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.323+0000 7f48e6ffd640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.323+0000 7f48f5d6d640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.323+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f48e80f0d40 con 0x7f48f0106f10 2024-09-12T02:48:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.324+0000 7f48f5d6d640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009890 tx=0x7f48ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.327+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e80bda60 con 0x7f48f0106f10 2024-09-12T02:48:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.544+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48f010a150 con 0x7f48f0106f10 2024-09-12T02:48:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.544+0000 7f48e6ffd640 1 -- 172.21.15.46:0/2700520470 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f48e80bd360 con 0x7f48f0106f10 2024-09-12T02:48:03.545 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T02:48:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 msgr2=0x7f48dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009890 tx=0x7f48ec008040 comp rx=0 tx=0).stop 2024-09-12T02:48:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 msgr2=0x7f48f019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009dc0 tx=0x7f48e8003ba0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 shutdown_connections 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48f0106f10 0x7f48f019cd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f0102790 0x7f48f01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 --2- 172.21.15.46:0/2700520470 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48f0101de0 0x7f48f0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 >> 172.21.15.46:0/2700520470 conn(0x7f48f00fd8c0 msgr2=0x7f48f00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 shutdown_connections 2024-09-12T02:48:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:03.547+0000 7f48f7fd1640 1 -- 172.21.15.46:0/2700520470 wait complete. 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:04.298 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T02:48:04.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T02:48:04.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T02:48:04.299 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T02:48:04.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:04.299 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-12T02:48:04.317 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T02:48:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T02:48:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T02:48:04.317 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42276sec preferred_lft 42276sec 2024-09-12T02:48:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-12T02:48:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:48:04.319 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:48:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:48:04.319 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-12T02:48:04.319 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T02:48:04.319 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:48:04.319 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:48:04.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-12T02:48:04.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-12T02:48:04.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-12T02:48:04.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T02:48:04.320 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:48:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:48:04.420 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:04 smithi046 ceph-mon[30951]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:04.420 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:04 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2700520470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:48:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:04 smithi196 ceph-mon[34587]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:04 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2700520470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:48:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:04.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:04.534 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T02:48:04.561 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-12T02:48:04.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:04.563 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-12T02:48:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:04 smithi145 ceph-mon[34674]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:04 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2700520470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:48:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-12T02:48:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T02:48:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T02:48:04.632 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T02:48:04.633 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T02:48:04.633 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:48:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:48:05.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:05.016 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-12T02:48:05.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:05.032 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-12T02:48:05.088 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:48:05.088 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:48:05.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:05.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T02:48:05.088 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:48:05.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:48:05.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:05.211 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T02:48:05.211 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:48:05.211 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:48:05.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T02:48:05.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T02:48:05.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T02:48:05.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T02:48:05.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T02:48:05.212 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:48:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:48:05.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:05.348 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:48:05.348 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:48:05.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-12T02:48:05.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T02:48:05.349 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:48:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:48:05.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:05.427 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T02:48:05.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:48:05.428 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:05.447 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T02:48:05.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:48:05.447 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:05.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-12T02:48:05.570 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T02:48:05.627 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-12T02:48:05.627 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-12T02:48:05.627 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-12T02:48:05.627 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-12T02:48:05.627 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-12T02:48:05.628 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f0c19eb0b94f9aac184e195c7f6002cc1352e1ada35b3e3c6c03ec4e6f9f0589/merged 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ad9fc8f688582ce4273a5548a52775c61cc23484a3d48f6497ef5c22e5130af/merged 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:05.629 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a27b6e2ef876def5568a0ef69271e77dbcc03bcf4664f6b949ed87a40de6b46b/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3a3cbbcb971750e1719fbacacda453e4a057ff24352230a8a3fd641caac9fe78/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/94a52bf189183ebdc3c84bdebaf8cb491f16f71cf57f6e443a7f1ac905c9458a/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ad996249b6177c62fc252717bde5789ead510efb33e13cac4ff07d9b8965a2ab/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/848ab539dead75324717f5809fdfd21e3d2c81405a32475f482fad361b374365/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/c00ac6eb3dc218102de5fbe213c6c2189702197b5132884c76a2d631dbb39741/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6e88b2d27417c36fe0f7cb64ed160327e936c22e5467e33c02b3d99461af6165/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a43b08778498f2caa9c2fc637ca3a82fbea959c9c67e179e391c322e38b77a1d/merged 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:05.630 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:05.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:05.631 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-12T02:48:05.683 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T02:48:05.684 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T02:48:05.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-12T02:48:05.751 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T02:48:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-12T02:48:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-12T02:48:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-12T02:48:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-12T02:48:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-12T02:48:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f0c19eb0b94f9aac184e195c7f6002cc1352e1ada35b3e3c6c03ec4e6f9f0589/merged 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ad9fc8f688582ce4273a5548a52775c61cc23484a3d48f6497ef5c22e5130af/merged 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a27b6e2ef876def5568a0ef69271e77dbcc03bcf4664f6b949ed87a40de6b46b/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3a3cbbcb971750e1719fbacacda453e4a057ff24352230a8a3fd641caac9fe78/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/94a52bf189183ebdc3c84bdebaf8cb491f16f71cf57f6e443a7f1ac905c9458a/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ad996249b6177c62fc252717bde5789ead510efb33e13cac4ff07d9b8965a2ab/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/848ab539dead75324717f5809fdfd21e3d2c81405a32475f482fad361b374365/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/c00ac6eb3dc218102de5fbe213c6c2189702197b5132884c76a2d631dbb39741/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6e88b2d27417c36fe0f7cb64ed160327e936c22e5467e33c02b3d99461af6165/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a43b08778498f2caa9c2fc637ca3a82fbea959c9c67e179e391c322e38b77a1d/merged 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:05.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:05.813 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-12T02:48:05.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-09-12T02:48:05.860+0000 7f4478860480 -1 init, newargv = 0x5593037cca30 newargc=15 2024-09-12T02:48:05.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[75250]: starting ceph client 2024-09-12T02:48:05.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[75250]: starting fuse 2024-09-12T02:48:06.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:06 smithi196 ceph-mon[34587]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:06 smithi046 ceph-mon[30951]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:06 smithi145 ceph-mon[34674]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:06.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-12T02:48:07.001 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-12T02:48:07.062 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T02:48:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f0c19eb0b94f9aac184e195c7f6002cc1352e1ada35b3e3c6c03ec4e6f9f0589/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ad9fc8f688582ce4273a5548a52775c61cc23484a3d48f6497ef5c22e5130af/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a27b6e2ef876def5568a0ef69271e77dbcc03bcf4664f6b949ed87a40de6b46b/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3a3cbbcb971750e1719fbacacda453e4a057ff24352230a8a3fd641caac9fe78/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/94a52bf189183ebdc3c84bdebaf8cb491f16f71cf57f6e443a7f1ac905c9458a/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ad996249b6177c62fc252717bde5789ead510efb33e13cac4ff07d9b8965a2ab/merged 2024-09-12T02:48:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/848ab539dead75324717f5809fdfd21e3d2c81405a32475f482fad361b374365/merged 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/c00ac6eb3dc218102de5fbe213c6c2189702197b5132884c76a2d631dbb39741/merged 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6e88b2d27417c36fe0f7cb64ed160327e936c22e5467e33c02b3d99461af6165/merged 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a43b08778498f2caa9c2fc637ca3a82fbea959c9c67e179e391c322e38b77a1d/merged 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:07.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:07.066 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-12T02:48:07.122 INFO:teuthology.orchestra.run.smithi046.stdout:120 2024-09-12T02:48:07.122 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:48:07.122 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T02:48:07.123 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-09-12T02:48:07.124 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-09-12T02:48:07.255 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.75250.asok 2024-09-12T02:48:07.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:07.267 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75250.asok 2024-09-12T02:48:07.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:07.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75250.asok status 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "75250", 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-09-12T02:48:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 14697, 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 14697 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 886554736 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T02:48:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 886554736 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.14697 192.168.0.1:0/886554736", 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/886554736", 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 17, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 88, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-09-12T02:48:07.410 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:48:07.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:07.417 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-09-12T02:48:07.417 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:07.434 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-09-12T02:48:07.434 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:07.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:07.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:07.501 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T02:48:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:48:07.629 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-09-12T02:48:07.629 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T02:48:07.630 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:07.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:48:07.636 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:48:07.636 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:48:07.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T02:48:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:08.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:08 smithi046 ceph-mon[30951]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- 172.21.15.46:0/2233761136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 msgr2=0x7f306c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 --2- 172.21.15.46:0/2233761136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c10bb50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f305c009900 tx=0x7f305c02a480 comp rx=0 tx=0).stop 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- 172.21.15.46:0/2233761136 shutdown_connections 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 --2- 172.21.15.46:0/2233761136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 0x7f306c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 --2- 172.21.15.46:0/2233761136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c10bb50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 --2- 172.21.15.46:0/2233761136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 0x7f306c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- 172.21.15.46:0/2233761136 >> 172.21.15.46:0/2233761136 conn(0x7f306c0fc7a0 msgr2=0x7f306c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- 172.21.15.46:0/2233761136 shutdown_connections 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- 172.21.15.46:0/2233761136 wait complete. 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 Processor -- start 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.451+0000 7f30745eb640 1 -- start start 2024-09-12T02:48:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 0x7f306c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 0x7f306c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f306c111ee0 con 0x7f306c103660 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f306c111d60 con 0x7f306c102d40 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f30745eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f306c112060 con 0x7f306c10c170 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55350/0 (socket says 172.21.15.46:55350) 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 -- 172.21.15.46:0/779134549 learned_addr learned my addr 172.21.15.46:0/779134549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3072387640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 0x7f306c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3072b88640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 0x7f306c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 -- 172.21.15.46:0/779134549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 msgr2=0x7f306c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 0x7f306c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 -- 172.21.15.46:0/779134549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 msgr2=0x7f306c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 0x7f306c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 -- 172.21.15.46:0/779134549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c1a69a0 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.452+0000 7f3071b86640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f305c006010 tx=0x7f305c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.453+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c03f030 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.453+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f305c003800 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.453+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1a6c30 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.453+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c035070 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.453+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1a7160 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.454+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306c1031a0 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.455+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f305c0039b0 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.455+0000 7f30637fe640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 0x7f30580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.456+0000 7f3072387640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 0x7f30580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.456+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f305c0efc80 con 0x7f306c103660 2024-09-12T02:48:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.456+0000 7f3072387640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 0x7f30580a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3068009990 tx=0x7f3068009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.459+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f305c0bc9a0 con 0x7f306c103660 2024-09-12T02:48:08.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:08 smithi196 ceph-mon[34587]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:08 smithi145 ceph-mon[34674]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:48:08.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.646+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f306c02d7c0 con 0x7f306c103660 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.646+0000 7f30637fe640 1 -- 172.21.15.46:0/779134549 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f305c0bc2a0 con 0x7f306c103660 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2936988098 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2423476506 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1895867717 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2351168704 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:08.647 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/829485206 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/791127312 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 msgr2=0x7f30580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 0x7f30580a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3068009990 tx=0x7f3068009110 comp rx=0 tx=0).stop 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 msgr2=0x7f306c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f305c006010 tx=0x7f305c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 shutdown_connections 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30580a2c00 0x7f30580a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f306c10c170 0x7f306c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c103660 0x7f306c100ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 --2- 172.21.15.46:0/779134549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f306c102d40 0x7f306c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 >> 172.21.15.46:0/779134549 conn(0x7f306c0fc7a0 msgr2=0x7f306c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 shutdown_connections 2024-09-12T02:48:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:08.648+0000 7f30745eb640 1 -- 172.21.15.46:0/779134549 wait complete. 2024-09-12T02:48:09.394 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:09 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/779134549' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:09.428 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:48:09.428 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:48:09.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T02:48:09.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:09 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/779134549' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:09.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:09 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/779134549' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:09.747 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 -- 172.21.15.46:0/3669311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678100e20 msgr2=0x7fd678101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 --2- 172.21.15.46:0/3669311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678100e20 0x7fd678101200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd660009900 tx=0x7fd66002a480 comp rx=0 tx=0).stop 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 -- 172.21.15.46:0/3669311382 shutdown_connections 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 --2- 172.21.15.46:0/3669311382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 0x7fd67810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 --2- 172.21.15.46:0/3669311382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678102570 0x7fd6781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 --2- 172.21.15.46:0/3669311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678100e20 0x7fd678101200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.284+0000 7fd67edcd640 1 -- 172.21.15.46:0/3669311382 >> 172.21.15.46:0/3669311382 conn(0x7fd6780fc7d0 msgr2=0x7fd6780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- 172.21.15.46:0/3669311382 shutdown_connections 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- 172.21.15.46:0/3669311382 wait complete. 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 Processor -- start 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- start start 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678100e20 0x7fd678199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:10.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 0x7fd6781a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110e00 con 0x7fd678102570 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110c80 con 0x7fd678100e20 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.285+0000 7fd67edcd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110f80 con 0x7fd678102ef0 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55362/0 (socket says 172.21.15.46:55362) 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 -- 172.21.15.46:0/1494504334 learned_addr learned my addr 172.21.15.46:0/1494504334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd67cb69640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678100e20 0x7fd678199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd67d36a640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 0x7fd6781a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 -- 172.21.15.46:0/1494504334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 msgr2=0x7fd6781a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 0x7fd6781a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 -- 172.21.15.46:0/1494504334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678100e20 msgr2=0x7fd678199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678100e20 0x7fd678199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 -- 172.21.15.46:0/1494504334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6781a47d0 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd677fff640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd66800db80 tx=0x7fd66800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668017070 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd668003b70 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6781a49a0 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.286+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680104c0 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.287+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a4ee0 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.288+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd678101200 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.288+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd668004020 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.289+0000 7fd675ffb640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 0x7fd6640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.289+0000 7fd67cb69640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 0x7fd6640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.289+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fd6680cf230 con 0x7fd678102570 2024-09-12T02:48:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.290+0000 7fd67cb69640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 0x7fd6640a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd660009230 tx=0x7fd6600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.293+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66809bff0 con 0x7fd678102570 2024-09-12T02:48:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:10 smithi196 ceph-mon[34587]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.482+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd678109090 con 0x7fd678102570 2024-09-12T02:48:10.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:10 smithi046 ceph-mon[30951]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.482+0000 7fd675ffb640 1 -- 172.21.15.46:0/1494504334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7fd66809b8f0 con 0x7fd678102570 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2936988098 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2423476506 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1895867717 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:10.483 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2351168704 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/829485206 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/791127312 2024-09-13T02:40:11.855373+0000 2024-09-12T02:48:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-09-12T02:48:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.485+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 msgr2=0x7fd6640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.485+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 0x7fd6640a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd660009230 tx=0x7fd6600058d0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.485+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 msgr2=0x7fd678199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.485+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd66800db80 tx=0x7fd66800b630 comp rx=0 tx=0).stop 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 shutdown_connections 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6640a2c00 0x7fd6640a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd678102ef0 0x7fd6781a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678102570 0x7fd678199750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 --2- 172.21.15.46:0/1494504334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678100e20 0x7fd678199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 >> 172.21.15.46:0/1494504334 conn(0x7fd6780fc7d0 msgr2=0x7fd6780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 shutdown_connections 2024-09-12T02:48:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:10.486+0000 7fd67edcd640 1 -- 172.21.15.46:0/1494504334 wait complete. 2024-09-12T02:48:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:10 smithi145 ceph-mon[34674]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:11.157 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-09-12T02:48:11.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:11.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T02:48:11.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[75250]: fuse finished with error 0 and tester_r 0 2024-09-12T02:48:11.187 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T02:48:11.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:48:11.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:48:11.242 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T02:48:11.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:48:11.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:11 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1494504334' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:11 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1494504334' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:11.565 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:11 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1494504334' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:48:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 -- 172.21.15.46:0/3438537035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874106f10 msgr2=0x7fd87410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/3438537035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874106f10 0x7fd87410ea90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009ca0 tx=0x7fd86c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 -- 172.21.15.46:0/3438537035 shutdown_connections 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/3438537035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874106f10 0x7fd87410ea90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/3438537035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874102790 0x7fd8741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/3438537035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874101de0 0x7fd8741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 -- 172.21.15.46:0/3438537035 >> 172.21.15.46:0/3438537035 conn(0x7fd8740fd8c0 msgr2=0x7fd8740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 -- 172.21.15.46:0/3438537035 shutdown_connections 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.073+0000 7fd87a7d8640 1 -- 172.21.15.46:0/3438537035 wait complete. 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 Processor -- start 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 -- start start 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874102790 0x7fd874198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874106f10 0x7fd87419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd874111e20 con 0x7fd874101de0 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd874111ca0 con 0x7fd874102790 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd87a7d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd874111fa0 con 0x7fd874106f10 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd873fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd873fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55382/0 (socket says 172.21.15.46:55382) 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.074+0000 7fd873fff640 1 -- 172.21.15.46:0/4174521988 learned_addr learned my addr 172.21.15.46:0/4174521988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd878d75640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874106f10 0x7fd87419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd8737fe640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874102790 0x7fd874198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 -- 172.21.15.46:0/4174521988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874106f10 msgr2=0x7fd87419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874106f10 0x7fd87419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 -- 172.21.15.46:0/4174521988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874102790 msgr2=0x7fd874198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874102790 0x7fd874198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 -- 172.21.15.46:0/4174521988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87419d2c0 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd873fff640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d950 tx=0x7fd86400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864017070 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87419d5b0 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd864009c30 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86400f620 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.075+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741a4e90 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.076+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8741021c0 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.078+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd8640109a0 con 0x7fd874101de0 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.078+0000 7fd8717fa640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 0x7fd85c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.078+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fd8640cee20 con 0x7fd874101de0 2024-09-12T02:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.078+0000 7fd8737fe640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 0x7fd85c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.079+0000 7fd8737fe640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 0x7fd85c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8740fdce0 tx=0x7fd86800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.083+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd864009de0 con 0x7fd874101de0 2024-09-12T02:48:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.263+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd874199670 con 0x7fd874101de0 2024-09-12T02:48:12.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:12 smithi046 ceph-mon[30951]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.265+0000 7fd8717fa640 1 -- 172.21.15.46:0/4174521988 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7fd86409b4f0 con 0x7fd874101de0 2024-09-12T02:48:12.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:12.266 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:48:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 msgr2=0x7fd85c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 0x7fd85c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8740fdce0 tx=0x7fd86800b390 comp rx=0 tx=0).stop 2024-09-12T02:48:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 msgr2=0x7fd874198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d950 tx=0x7fd86400de10 comp rx=0 tx=0).stop 2024-09-12T02:48:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 shutdown_connections 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd85c0a2b80 0x7fd85c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874106f10 0x7fd87419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd874102790 0x7fd874198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 --2- 172.21.15.46:0/4174521988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd874101de0 0x7fd874198310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 >> 172.21.15.46:0/4174521988 conn(0x7fd8740fd8c0 msgr2=0x7fd8740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 shutdown_connections 2024-09-12T02:48:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:12.267+0000 7fd87a7d8640 1 -- 172.21.15.46:0/4174521988 wait complete. 2024-09-12T02:48:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:12 smithi196 ceph-mon[34587]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:12 smithi145 ceph-mon[34674]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-12T02:48:13.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:48:13.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:13 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4174521988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:13.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:13 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4174521988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:13 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4174521988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 -- 172.21.15.46:0/4066146983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3088102b50 msgr2=0x7f3088100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 --2- 172.21.15.46:0/4066146983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3088102b50 0x7f3088100090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3084009900 tx=0x7f308402a400 comp rx=0 tx=0).stop 2024-09-12T02:48:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 -- 172.21.15.46:0/4066146983 shutdown_connections 2024-09-12T02:48:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 --2- 172.21.15.46:0/4066146983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308810e810 0x7f3088110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 --2- 172.21.15.46:0/4066146983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102fc0 0x7f30881005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 --2- 172.21.15.46:0/4066146983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3088102b50 0x7f3088100090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 -- 172.21.15.46:0/4066146983 >> 172.21.15.46:0/4066146983 conn(0x7f3088077210 msgr2=0x7f30880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.791+0000 7f308fedf640 1 -- 172.21.15.46:0/4066146983 shutdown_connections 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 -- 172.21.15.46:0/4066146983 wait complete. 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 Processor -- start 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 -- start start 2024-09-12T02:48:13.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102b50 0x7f308819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308819ee50 0x7f308819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30881140e0 con 0x7f308810e810 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3088113f60 con 0x7f3088102b50 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308fedf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3088114260 con 0x7f308819ee50 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308d47a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308d47a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55394/0 (socket says 172.21.15.46:55394) 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.792+0000 7f308d47a640 1 -- 172.21.15.46:0/246559965 learned_addr learned my addr 172.21.15.46:0/246559965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308dc7b640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102b50 0x7f308819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308e47c640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308819ee50 0x7f308819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 -- 172.21.15.46:0/246559965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308819ee50 msgr2=0x7f308819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308819ee50 0x7f308819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 -- 172.21.15.46:0/246559965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102b50 msgr2=0x7f308819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102b50 0x7f308819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 -- 172.21.15.46:0/246559965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308819f910 con 0x7f308810e810 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308d47a640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f307800d740 tx=0x7f307800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078017070 con 0x7f308810e810 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f307800de50 con 0x7f308810e810 2024-09-12T02:48:13.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308819fc00 con 0x7f308810e810 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307800f620 con 0x7f308810e810 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.793+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30881a74b0 con 0x7f308810e810 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.794+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3088100090 con 0x7f308810e810 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.796+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f307801a070 con 0x7f308810e810 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.797+0000 7f307effd640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 0x7f30740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.797+0000 7f308dc7b640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 0x7f30740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:13.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.797+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f30780cec40 con 0x7f308810e810 2024-09-12T02:48:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.797+0000 7f308dc7b640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 0x7f30740a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3084031440 tx=0x7f3084005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:13.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.800+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f307809b9e0 con 0x7f308810e810 2024-09-12T02:48:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.976+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30881031a0 con 0x7f308810e810 2024-09-12T02:48:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.978+0000 7f307effd640 1 -- 172.21.15.46:0/246559965 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7f307809b2e0 con 0x7f308810e810 2024-09-12T02:48:13.979 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:13.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 msgr2=0x7f30740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 0x7f30740a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3084031440 tx=0x7f3084005b50 comp rx=0 tx=0).stop 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 msgr2=0x7f308819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f307800d740 tx=0x7f307800dc00 comp rx=0 tx=0).stop 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 shutdown_connections 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f30740a2c00 0x7f30740a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f308819ee50 0x7f308819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f308810e810 0x7f308819aac0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 --2- 172.21.15.46:0/246559965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3088102b50 0x7f308819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 >> 172.21.15.46:0/246559965 conn(0x7f3088077210 msgr2=0x7f308810fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 shutdown_connections 2024-09-12T02:48:13.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:13.980+0000 7f308fedf640 1 -- 172.21.15.46:0/246559965 wait complete. 2024-09-12T02:48:14.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:14 smithi046 ceph-mon[30951]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:14.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:14 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/246559965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:14 smithi196 ceph-mon[34587]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:14.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:14 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/246559965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:14 smithi145 ceph-mon[34674]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:14 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/246559965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:14.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T02:48:15.049 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 -- 172.21.15.46:0/2623946091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 msgr2=0x7faa8010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 --2- 172.21.15.46:0/2623946091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa8010d950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa78009930 tx=0x7faa7802a420 comp rx=0 tx=0).stop 2024-09-12T02:48:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 -- 172.21.15.46:0/2623946091 shutdown_connections 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 --2- 172.21.15.46:0/2623946091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa8010d950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 --2- 172.21.15.46:0/2623946091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 0x7faa80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 --2- 172.21.15.46:0/2623946091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa80106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 -- 172.21.15.46:0/2623946091 >> 172.21.15.46:0/2623946091 conn(0x7faa800fc7a0 msgr2=0x7faa800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 -- 172.21.15.46:0/2623946091 shutdown_connections 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.557+0000 7faa84e96640 1 -- 172.21.15.46:0/2623946091 wait complete. 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 Processor -- start 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 -- start start 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 0x7faa801991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa801a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.558+0000 7faa84e96640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110d50 con 0x7faa80101420 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa84e96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110bd0 con 0x7faa80106a80 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa84e96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110ed0 con 0x7faa80100a80 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48926/0 (socket says 172.21.15.46:48926) 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 -- 172.21.15.46:0/2530537062 learned_addr learned my addr 172.21.15.46:0/2530537062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7ed9d640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa801a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7e59c640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 0x7faa801991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 -- 172.21.15.46:0/2530537062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 msgr2=0x7faa801991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 0x7faa801991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 -- 172.21.15.46:0/2530537062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 msgr2=0x7faa801a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa801a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:15.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 -- 172.21.15.46:0/2530537062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa801a47a0 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7ed9d640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa801a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa7dd9b640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b4a0 tx=0x7faa7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74014070 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa74010ac0 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801a4970 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74017c30 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.559+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a4e30 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.560+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa80100ee0 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.562+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa74017070 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.562+0000 7faa6f7fe640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.562+0000 7faa7e59c640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.562+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7faa740ce740 con 0x7faa80101420 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.563+0000 7faa7e59c640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 0x7faa680a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa70007350 tx=0x7faa700072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.567+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa7409b400 con 0x7faa80101420 2024-09-12T02:48:15.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:15.779+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faa80109010 con 0x7faa80101420 2024-09-12T02:48:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.196+0000 7faa6f7fe640 1 -- 172.21.15.46:0/2530537062 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7faa7409ad00 con 0x7faa80101420 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 msgr2=0x7faa680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 0x7faa680a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa70007350 tx=0x7faa700072e0 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 msgr2=0x7faa80199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b4a0 tx=0x7faa7400b960 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 shutdown_connections 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa80106a80 0x7faa801a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa80101420 0x7faa80199720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 --2- 172.21.15.46:0/2530537062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa80100a80 0x7faa801991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 >> 172.21.15.46:0/2530537062 conn(0x7faa800fc7a0 msgr2=0x7faa800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 shutdown_connections 2024-09-12T02:48:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:16.198+0000 7faa84e96640 1 -- 172.21.15.46:0/2530537062 wait complete. 2024-09-12T02:48:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:16 smithi196 ceph-mon[34587]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:16 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:48:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:16 smithi046 ceph-mon[30951]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:16 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:48:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:16 smithi145 ceph-mon[34674]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:16 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:48:17.031 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-12T02:48:17.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T02:48:17.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:17 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:48:17.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:17 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:17.298 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:17.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:17 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:48:17.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:17 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:17 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2530537062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:48:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:17 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- 172.21.15.46:0/2275511617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0100e20 msgr2=0x7fbde0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 --2- 172.21.15.46:0/2275511617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0100e20 0x7fbde0101200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009900 tx=0x7fbdd002a480 comp rx=0 tx=0).stop 2024-09-12T02:48:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- 172.21.15.46:0/2275511617 shutdown_connections 2024-09-12T02:48:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 --2- 172.21.15.46:0/2275511617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0102ef0 0x7fbde010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 --2- 172.21.15.46:0/2275511617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102570 0x7fbde01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 --2- 172.21.15.46:0/2275511617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0100e20 0x7fbde0101200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- 172.21.15.46:0/2275511617 >> 172.21.15.46:0/2275511617 conn(0x7fbde00fc7d0 msgr2=0x7fbde00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- 172.21.15.46:0/2275511617 shutdown_connections 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- 172.21.15.46:0/2275511617 wait complete. 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 Processor -- start 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.801+0000 7fbde7255640 1 -- start start 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0100e20 0x7fbde0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 0x7fbde0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110e50 con 0x7fbde0102570 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110cd0 con 0x7fbde0100e20 2024-09-12T02:48:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde7255640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110fd0 con 0x7fbde0102ef0 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48950/0 (socket says 172.21.15.46:48950) 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 -- 172.21.15.46:0/451331288 learned_addr learned my addr 172.21.15.46:0/451331288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde4ff1640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0100e20 0x7fbde0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde57f2640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 0x7fbde0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 -- 172.21.15.46:0/451331288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 msgr2=0x7fbde0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 0x7fbde0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 -- 172.21.15.46:0/451331288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0100e20 msgr2=0x7fbde0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0100e20 0x7fbde0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 -- 172.21.15.46:0/451331288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde01a6980 con 0x7fbde0102570 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbde57f2640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 0x7fbde0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbddffff640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400b4a0 tx=0x7fbdd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:17.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.802+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.803+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd4010ac0 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.803+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde01a6c70 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.803+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4017c30 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.803+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a7160 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.804+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0101200 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.805+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbdd4017070 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.805+0000 7fbdddffb640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 0x7fbdc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.806+0000 7fbde4ff1640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 0x7fbdc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.806+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fbdd40ce630 con 0x7fbde0102570 2024-09-12T02:48:17.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.806+0000 7fbde4ff1640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 0x7fbdc80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009230 tx=0x7fbdd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:17.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:17.810+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdd409b350 con 0x7fbde0102570 2024-09-12T02:48:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.021+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbde0109090 con 0x7fbde0102570 2024-09-12T02:48:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.207+0000 7fbdddffb640 1 -- 172.21.15.46:0/451331288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fbdd409ac50 con 0x7fbde0102570 2024-09-12T02:48:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.209+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 msgr2=0x7fbdc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.209+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 0x7fbdc80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009230 tx=0x7fbdd00058d0 comp rx=0 tx=0).stop 2024-09-12T02:48:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.209+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 msgr2=0x7fbde0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.209+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400b4a0 tx=0x7fbdd400b960 comp rx=0 tx=0).stop 2024-09-12T02:48:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 shutdown_connections 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbdc80a2b80 0x7fbdc80a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde0102ef0 0x7fbde0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0102570 0x7fbde0197650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 --2- 172.21.15.46:0/451331288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbde0100e20 0x7fbde0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 >> 172.21.15.46:0/451331288 conn(0x7fbde00fc7d0 msgr2=0x7fbde00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 shutdown_connections 2024-09-12T02:48:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:18.210+0000 7fbde7255640 1 -- 172.21.15.46:0/451331288 wait complete. 2024-09-12T02:48:18.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:18 smithi046 ceph-mon[30951]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:18.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:18 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:48:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:18 smithi196 ceph-mon[34587]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:18 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:48:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:18 smithi145 ceph-mon[34674]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:18 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T02:48:18.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T02:48:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:19.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:48:19.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:48:19.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:48:19.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:19.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T02:48:19.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:19 smithi196 ceph-mon[34587]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T02:48:19.511 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:48:19.512 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:48:19.512 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:48:19.512 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:19.512 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T02:48:19.512 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:19 smithi145 ceph-mon[34674]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T02:48:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:48:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/451331288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T02:48:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:48:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:48:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T02:48:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:19 smithi046 ceph-mon[30951]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T02:48:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 -- 172.21.15.46:0/797560370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0106a80 msgr2=0x7f39e0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/797560370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0106a80 0x7f39e0106e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009930 tx=0x7f39dc02a420 comp rx=0 tx=0).stop 2024-09-12T02:48:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 -- 172.21.15.46:0/797560370 shutdown_connections 2024-09-12T02:48:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/797560370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0101420 0x7f39e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/797560370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 0x7f39e0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/797560370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0106a80 0x7f39e0106e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 -- 172.21.15.46:0/797560370 >> 172.21.15.46:0/797560370 conn(0x7f39e00fc7a0 msgr2=0x7f39e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 -- 172.21.15.46:0/797560370 shutdown_connections 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.768+0000 7f39e7e9d640 1 -- 172.21.15.46:0/797560370 wait complete. 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 Processor -- start 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 -- start start 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 0x7f39e00fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0106a80 0x7f39e00fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0110dd0 con 0x7f39e0101420 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0110c50 con 0x7f39e0106a80 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e7e9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0110f50 con 0x7f39e0100a80 2024-09-12T02:48:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e5438640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e5c39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 0x7f39e00fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e5438640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48970/0 (socket says 172.21.15.46:48970) 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e5438640 1 -- 172.21.15.46:0/3266259766 learned_addr learned my addr 172.21.15.46:0/3266259766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.769+0000 7f39e643a640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0106a80 0x7f39e00fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 -- 172.21.15.46:0/3266259766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 msgr2=0x7f39e00fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 0x7f39e00fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 -- 172.21.15.46:0/3266259766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0106a80 msgr2=0x7f39e00fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0106a80 0x7f39e00fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 -- 172.21.15.46:0/3266259766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01a69c0 con 0x7f39e0101420 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e5438640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b4a0 tx=0x7f39d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0014070 con 0x7f39e0101420 2024-09-12T02:48:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39d0010ac0 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01a6cb0 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0017c30 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.770+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a71a0 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.771+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0100f60 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.772+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39d0017070 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.773+0000 7f39d6ffd640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 0x7f39cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.773+0000 7f39e5c39640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 0x7f39cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.773+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f39d00ce5a0 con 0x7f39e0101420 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.774+0000 7f39e5c39640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 0x7f39cc0a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39dc006010 tx=0x7f39dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.778+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39d009b2c0 con 0x7f39e0101420 2024-09-12T02:48:19.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:19.991+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f39e010a000 con 0x7f39e0101420 2024-09-12T02:48:20.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.097+0000 7f39d6ffd640 1 -- 172.21.15.46:0/3266259766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f39d009abc0 con 0x7f39e0101420 2024-09-12T02:48:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 msgr2=0x7f39cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 0x7f39cc0a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39dc006010 tx=0x7f39dc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:48:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 msgr2=0x7f39e00fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b4a0 tx=0x7f39d000b960 comp rx=0 tx=0).stop 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 shutdown_connections 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39e0106a80 0x7f39e00fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e0101420 0x7f39e00fd3b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f39cc0a2c00 0x7f39cc0a50c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 --2- 172.21.15.46:0/3266259766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e0100a80 0x7f39e00fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 >> 172.21.15.46:0/3266259766 conn(0x7f39e00fc7a0 msgr2=0x7f39e006abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 shutdown_connections 2024-09-12T02:48:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:20.100+0000 7f39e7e9d640 1 -- 172.21.15.46:0/3266259766 wait complete. 2024-09-12T02:48:20.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:active 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: Cluster is now healthy 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:active 2024-09-12T02:48:20.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:20 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:active 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: Cluster is now healthy 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:48:20.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:48:20.475 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:48:20.475 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:active 2024-09-12T02:48:20.475 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:20 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:20.530 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] up:active 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: Cluster is now healthy 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3266259766' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] up:active 2024-09-12T02:48:20.531 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:20 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:20.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T02:48:21.064 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 -- 172.21.15.46:0/3551941329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4106f10 msgr2=0x7fb9c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/3551941329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4106f10 0x7fb9c410ea90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009ca0 tx=0x7fb9bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 -- 172.21.15.46:0/3551941329 shutdown_connections 2024-09-12T02:48:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/3551941329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4106f10 0x7fb9c410ea90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/3551941329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102790 0x7fb9c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/3551941329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4101de0 0x7fb9c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 -- 172.21.15.46:0/3551941329 >> 172.21.15.46:0/3551941329 conn(0x7fb9c40fd8c0 msgr2=0x7fb9c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 -- 172.21.15.46:0/3551941329 shutdown_connections 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.663+0000 7fb9cafa8640 1 -- 172.21.15.46:0/3551941329 wait complete. 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 Processor -- start 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 -- start start 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4102790 0x7fb9c4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4106f10 0x7fb9c419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111e70 con 0x7fb9c4101de0 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111cf0 con 0x7fb9c4102790 2024-09-12T02:48:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9cafa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111ff0 con 0x7fb9c4106f10 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9c8d44640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9c8d44640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48988/0 (socket says 172.21.15.46:48988) 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9c8d44640 1 -- 172.21.15.46:0/2206887911 learned_addr learned my addr 172.21.15.46:0/2206887911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9bbfff640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4102790 0x7fb9c4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.664+0000 7fb9c9545640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4106f10 0x7fb9c419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 -- 172.21.15.46:0/2206887911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4106f10 msgr2=0x7fb9c419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4106f10 0x7fb9c419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 -- 172.21.15.46:0/2206887911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4102790 msgr2=0x7fb9c4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4102790 0x7fb9c4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 -- 172.21.15.46:0/2206887911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419d2c0 con 0x7fb9c4101de0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9c8d44640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000cd80 con 0x7fb9c4101de0 2024-09-12T02:48:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c00108e0 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419d5b0 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0018920 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.665+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41a4e90 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.666+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c41021c0 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.668+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9c0010a90 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.668+0000 7fb9b9ffb640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 0x7fb9ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.668+0000 7fb9bbfff640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 0x7fb9ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.668+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb9c00cf550 con 0x7fb9c4101de0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.669+0000 7fb9bbfff640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 0x7fb9ac0a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40fdce0 tx=0x7fb9b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.673+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9c009bc20 con 0x7fb9c4101de0 2024-09-12T02:48:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:21.887+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb9c4199670 con 0x7fb9c4101de0 2024-09-12T02:48:22.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.225+0000 7fb9b9ffb640 1 -- 172.21.15.46:0/2206887911 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fb9c00860b0 con 0x7fb9c4101de0 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 msgr2=0x7fb9ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 0x7fb9ac0a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40fdce0 tx=0x7fb9b400b390 comp rx=0 tx=0).stop 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 msgr2=0x7fb9c4198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 shutdown_connections 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2b80 0x7fb9ac0a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4106f10 0x7fb9c419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4102790 0x7fb9c4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 --2- 172.21.15.46:0/2206887911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4101de0 0x7fb9c4198310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 >> 172.21.15.46:0/2206887911 conn(0x7fb9c40fd8c0 msgr2=0x7fb9c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 shutdown_connections 2024-09-12T02:48:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:22.228+0000 7fb9cafa8640 1 -- 172.21.15.46:0/2206887911 wait complete. 2024-09-12T02:48:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:22 smithi046 ceph-mon[30951]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-12T02:48:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:22 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:48:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:22 smithi196 ceph-mon[34587]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-12T02:48:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:22 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:48:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:22 smithi145 ceph-mon[34674]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-12T02:48:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:22 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:48:22.927 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-12T02:48:22.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T02:48:22.970 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:48:22 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: 2024-09-12T02:48:22.602+0000 7f626d396640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T02:48:22.970 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 02:48:22 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: 2024-09-12T02:48:22.602+0000 7f626d396640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T02:48:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:23.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:48:23.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:23.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-mon[30951]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:48:23.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:23 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:48:23.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:23 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:23.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:23 smithi145 ceph-mon[34674]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 -- 172.21.15.46:0/99843610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 msgr2=0x7fc290103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 --2- 172.21.15.46:0/99843610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290103170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc280009930 tx=0x7fc28002a420 comp rx=0 tx=0).stop 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 -- 172.21.15.46:0/99843610 shutdown_connections 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 --2- 172.21.15.46:0/99843610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290108290 0x7fc29010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 --2- 172.21.15.46:0/99843610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290103740 0x7fc290107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 --2- 172.21.15.46:0/99843610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290103170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 -- 172.21.15.46:0/99843610 >> 172.21.15.46:0/99843610 conn(0x7fc2900fc7d0 msgr2=0x7fc2900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 -- 172.21.15.46:0/99843610 shutdown_connections 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.698+0000 7fc295cb1640 1 -- 172.21.15.46:0/99843610 wait complete. 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 Processor -- start 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 -- start start 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290103740 0x7fc290198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290108290 0x7fc29019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc290111fa0 con 0x7fc290102d90 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc290111e20 con 0x7fc290108290 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc295cb1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc290112120 con 0x7fc290103740 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49008/0 (socket says 172.21.15.46:49008) 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc28f7fe640 1 -- 172.21.15.46:0/601931195 learned_addr learned my addr 172.21.15.46:0/601931195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.699+0000 7fc28ffff640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290108290 0x7fc29019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28effd640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290103740 0x7fc290198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 -- 172.21.15.46:0/601931195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290103740 msgr2=0x7fc290198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290103740 0x7fc290198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 -- 172.21.15.46:0/601931195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290108290 msgr2=0x7fc29019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290108290 0x7fc29019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 -- 172.21.15.46:0/601931195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29019d530 con 0x7fc290102d90 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28f7fe640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc280005940 tx=0x7fc2800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28003f030 con 0x7fc290102d90 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29019d7c0 con 0x7fc290102d90 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc280003cb0 con 0x7fc290102d90 2024-09-12T02:48:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280035070 con 0x7fc290102d90 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.700+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2901a5100 con 0x7fc290102d90 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.701+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29006a7f0 con 0x7fc290102d90 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.703+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc280042030 con 0x7fc290102d90 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.703+0000 7fc28cff9640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 0x7fc2780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.703+0000 7fc28effd640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 0x7fc2780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.703+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc2800ef980 con 0x7fc290102d90 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.704+0000 7fc28effd640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 0x7fc2780a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2900fef20 tx=0x7fc284009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.707+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2800bc6a0 con 0x7fc290102d90 2024-09-12T02:48:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:23 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2206887911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:48:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:23 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:48:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:23 smithi196 ceph-mon[34587]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-12T02:48:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.909+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc290062630 con 0x7fc2780a2b80 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.933+0000 7fc28cff9640 1 -- 172.21.15.46:0/601931195 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc290062630 con 0x7fc2780a2b80 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 msgr2=0x7fc2780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 0x7fc2780a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2900fef20 tx=0x7fc284009110 comp rx=0 tx=0).stop 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 msgr2=0x7fc290198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc280005940 tx=0x7fc2800042f0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 shutdown_connections 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc290108290 0x7fc29019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2780a2b80 0x7fc2780a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc290103740 0x7fc290198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 --2- 172.21.15.46:0/601931195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290102d90 0x7fc290198580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.935+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 >> 172.21.15.46:0/601931195 conn(0x7fc2900fc7d0 msgr2=0x7fc290105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.936+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 shutdown_connections 2024-09-12T02:48:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:23.936+0000 7fc295cb1640 1 -- 172.21.15.46:0/601931195 wait complete. 2024-09-12T02:48:24.237 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:48:23.915+0000 7f0a83f25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:48:24.237 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:48:23.915+0000 7f0a83f25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:48:24.237 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:48:23.915+0000 7f0a83f25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:48:24.237 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:48:23.915+0000 7f0a83f25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:48:24.237 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:48:23 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:48:23.915+0000 7f0a83f25640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:48:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:24 smithi046 ceph-mon[30951]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:24 smithi145 ceph-mon[34674]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:24 smithi196 ceph-mon[34587]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:24.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T02:48:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:25.431 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:25 smithi046 ceph-mon[30951]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:48:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 -- 172.21.15.46:0/3388023078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 msgr2=0x7f64f8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/3388023078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f8104e80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009900 tx=0x7f64e002a410 comp rx=0 tx=0).stop 2024-09-12T02:48:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 -- 172.21.15.46:0/3388023078 shutdown_connections 2024-09-12T02:48:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/3388023078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/3388023078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f8104e80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/3388023078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 0x7f64f8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 -- 172.21.15.46:0/3388023078 >> 172.21.15.46:0/3388023078 conn(0x7f64f8077210 msgr2=0x7f64f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 -- 172.21.15.46:0/3388023078 shutdown_connections 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.518+0000 7f64fd3d7640 1 -- 172.21.15.46:0/3388023078 wait complete. 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 Processor -- start 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 -- start start 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 0x7f64f80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8110d60 con 0x7f64f8100e30 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8110be0 con 0x7f64f8100480 2024-09-12T02:48:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64fd3d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8110ee0 con 0x7f64f81055b0 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64effff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64effff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48382/0 (socket says 172.21.15.46:48382) 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64effff640 1 -- 172.21.15.46:0/787510260 learned_addr learned my addr 172.21.15.46:0/787510260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.519+0000 7f64f6ffd640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 0x7f64f80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64f77fe640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 -- 172.21.15.46:0/787510260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 msgr2=0x7f64f80fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f80fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 -- 172.21.15.46:0/787510260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 msgr2=0x7f64f80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 0x7f64f80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 -- 172.21.15.46:0/787510260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81a68c0 con 0x7f64f8100e30 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64f77fe640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64effff640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005940 tx=0x7f64e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e003f030 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e0003aa0 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f81a6b50 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0035070 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.520+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f81a7080 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.521+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f8100860 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.522+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f64e0042030 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.523+0000 7f64f4ff9640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 0x7f64e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.523+0000 7f64f6ffd640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 0x7f64e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.524+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f64e00efb40 con 0x7f64f8100e30 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.524+0000 7f64f6ffd640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 0x7f64e40a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005d00 tx=0x7f64e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.528+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64e00bc700 con 0x7f64f8100e30 2024-09-12T02:48:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:25 smithi145 ceph-mon[34674]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:48:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:25 smithi196 ceph-mon[34587]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T02:48:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.729+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f64f81045c0 con 0x7f64e40a2c90 2024-09-12T02:48:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.753+0000 7f64f4ff9640 1 -- 172.21.15.46:0/787510260 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64f81045c0 con 0x7f64e40a2c90 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 msgr2=0x7f64e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 0x7f64e40a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005d00 tx=0x7f64e800b410 comp rx=0 tx=0).stop 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 msgr2=0x7f64f80fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005940 tx=0x7f64e0004270 comp rx=0 tx=0).stop 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 shutdown_connections 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f64e40a2c90 0x7f64e40a5150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f81055b0 0x7f64f80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8100e30 0x7f64f80fcbd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 --2- 172.21.15.46:0/787510260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64f8100480 0x7f64f80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 >> 172.21.15.46:0/787510260 conn(0x7f64f8077210 msgr2=0x7f64f810dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 shutdown_connections 2024-09-12T02:48:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:25.755+0000 7f64fd3d7640 1 -- 172.21.15.46:0/787510260 wait complete. 2024-09-12T02:48:26.446 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:26 smithi046 ceph-mon[30951]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:48:26.446 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:26 smithi046 ceph-mon[30951]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:26.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T02:48:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:26 smithi145 ceph-mon[34674]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:48:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:26 smithi145 ceph-mon[34674]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:26 smithi196 ceph-mon[34587]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:48:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:26 smithi196 ceph-mon[34587]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 -- 172.21.15.46:0/4201032691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 msgr2=0x7f10601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 --2- 172.21.15.46:0/4201032691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f10601067e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1048009900 tx=0x7f104802a410 comp rx=0 tx=0).stop 2024-09-12T02:48:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 -- 172.21.15.46:0/4201032691 shutdown_connections 2024-09-12T02:48:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 --2- 172.21.15.46:0/4201032691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 0x7f106010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 --2- 172.21.15.46:0/4201032691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f10601067e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 --2- 172.21.15.46:0/4201032691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 0x7f10601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 -- 172.21.15.46:0/4201032691 >> 172.21.15.46:0/4201032691 conn(0x7f10600fd8c0 msgr2=0x7f10600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 -- 172.21.15.46:0/4201032691 shutdown_connections 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.382+0000 7f1065b23640 1 -- 172.21.15.46:0/4201032691 wait complete. 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 Processor -- start 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 -- start start 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 0x7f10600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 0x7f1060074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1060111e40 con 0x7f1060102790 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1060111cc0 con 0x7f1060106f10 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f1065b23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1060111fc0 con 0x7f1060101de0 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f105effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f105effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48410/0 (socket says 172.21.15.46:48410) 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.383+0000 7f105effd640 1 -- 172.21.15.46:0/4273664030 learned_addr learned my addr 172.21.15.46:0/4273664030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105ffff640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 0x7f1060074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105f7fe640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 0x7f10600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 -- 172.21.15.46:0/4273664030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 msgr2=0x7f10600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 0x7f10600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 -- 172.21.15.46:0/4273664030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 msgr2=0x7f1060074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 0x7f1060074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 -- 172.21.15.46:0/4273664030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10601a6880 con 0x7f1060102790 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105effd640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1048006010 tx=0x7f1048004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104803f030 con 0x7f1060102790 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601a6b10 con 0x7f1060102790 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10480035f0 con 0x7f1060102790 2024-09-12T02:48:27.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048035070 con 0x7f1060102790 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.384+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a6ff0 con 0x7f1060102790 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.385+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10601021c0 con 0x7f1060102790 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.387+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1048003b10 con 0x7f1060102790 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.387+0000 7f105cff9640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 0x7f104c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.388+0000 7f105f7fe640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 0x7f104c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.388+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f10480f0040 con 0x7f1060102790 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.388+0000 7f105f7fe640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 0x7f104c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10500058c0 tx=0x7f105000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:27.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.392+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10480bcc00 con 0x7f1060102790 2024-09-12T02:48:27.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:27 smithi046 ceph-mon[30951]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:48:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.596+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f106010a1c0 con 0x7f104c0a2c10 2024-09-12T02:48:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.602+0000 7f105cff9640 1 -- 172.21.15.46:0/4273664030 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f105000ea90 con 0x7f104c0a2c10 2024-09-12T02:48:27.602 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/7124dae5-f95d-40d9-b3df-bb43c74cb3cf 2024-09-12T02:48:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 msgr2=0x7f104c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 0x7f104c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10500058c0 tx=0x7f105000a4c0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 msgr2=0x7f1060074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1048006010 tx=0x7f1048004270 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 shutdown_connections 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1060106f10 0x7f1060074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1060102790 0x7f1060074a30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f104c0a2c10 0x7f104c0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 --2- 172.21.15.46:0/4273664030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1060101de0 0x7f10600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 >> 172.21.15.46:0/4273664030 conn(0x7f10600fd8c0 msgr2=0x7f10600fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 shutdown_connections 2024-09-12T02:48:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:27.604+0000 7f1065b23640 1 -- 172.21.15.46:0/4273664030 wait complete. 2024-09-12T02:48:27.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:27 smithi145 ceph-mon[34674]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:48:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:27 smithi196 ceph-mon[34587]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T02:48:28.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T02:48:28.485 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:28.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:28 smithi046 ceph-mon[30951]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:28 smithi145 ceph-mon[34674]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:28 smithi196 ceph-mon[34587]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- 172.21.15.46:0/2695001003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca5410e3c0 msgr2=0x7fca54110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2695001003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca5410e3c0 0x7fca54110810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009ca0 tx=0x7fca4c02a950 comp rx=0 tx=0).stop 2024-09-12T02:48:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- 172.21.15.46:0/2695001003 shutdown_connections 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2695001003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca5410e3c0 0x7fca54110810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2695001003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074f40 0x7fca5410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2695001003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca54074620 0x7fca54074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- 172.21.15.46:0/2695001003 >> 172.21.15.46:0/2695001003 conn(0x7fca540fd8c0 msgr2=0x7fca540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- 172.21.15.46:0/2695001003 shutdown_connections 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- 172.21.15.46:0/2695001003 wait complete. 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 Processor -- start 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.082+0000 7fca5bb95640 1 -- start start 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074620 0x7fca5419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 0x7fca5419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca541140b0 con 0x7fca54074f40 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fca54113f30 con 0x7fca5410e3c0 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5bb95640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca54114230 con 0x7fca54074620 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48428/0 (socket says 172.21.15.46:48428) 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 -- 172.21.15.46:0/2417638602 learned_addr learned my addr 172.21.15.46:0/2417638602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5a132640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 0x7fca5419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59931640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074620 0x7fca5419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 -- 172.21.15.46:0/2417638602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074620 msgr2=0x7fca5419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074620 0x7fca5419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 -- 172.21.15.46:0/2417638602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 msgr2=0x7fca5419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 0x7fca5419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 -- 172.21.15.46:0/2417638602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5419f520 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca5a132640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 0x7fca5419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca59130640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b4a0 tx=0x7fca4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.083+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44014070 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.084+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5419f810 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.084+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44010ac0 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.084+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44017c30 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.084+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541a70f0 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.085+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca54074a00 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.086+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca44017070 con 0x7fca54074f40 2024-09-12T02:48:29.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.086+0000 7fca4affd640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 0x7fca400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:29.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.087+0000 7fca59931640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 0x7fca400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:29.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.087+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fca440ce7a0 con 0x7fca54074f40 2024-09-12T02:48:29.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.087+0000 7fca59931640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 0x7fca400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca500058c0 tx=0x7fca5000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:29.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.091+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca4409b470 con 0x7fca54074f40 2024-09-12T02:48:29.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.296+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fca54062630 con 0x7fca400a2c10 2024-09-12T02:48:29.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.319+0000 7fca4affd640 1 -- 172.21.15.46:0/2417638602 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca54062630 con 0x7fca400a2c10 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 msgr2=0x7fca400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 0x7fca400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca500058c0 tx=0x7fca5000a4c0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 msgr2=0x7fca5419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b4a0 tx=0x7fca4400b960 comp rx=0 tx=0).stop 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 shutdown_connections 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca5410e3c0 0x7fca5419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca54074f40 0x7fca5419aab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fca400a2c10 0x7fca400a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 --2- 172.21.15.46:0/2417638602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca54074620 0x7fca5419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 >> 172.21.15.46:0/2417638602 conn(0x7fca540fd8c0 msgr2=0x7fca540fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 shutdown_connections 2024-09-12T02:48:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:29.321+0000 7fca5bb95640 1 -- 172.21.15.46:0/2417638602 wait complete. 2024-09-12T02:48:29.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:29 smithi046 ceph-mon[30951]: pgmap v314: 129 pgs: 129 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:29.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:29 smithi145 ceph-mon[34674]: pgmap v314: 129 pgs: 129 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:29 smithi196 ceph-mon[34587]: pgmap v314: 129 pgs: 129 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:29.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T02:48:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:30.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:30 smithi046 ceph-mon[30951]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:30 smithi145 ceph-mon[34674]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:30 smithi196 ceph-mon[34587]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 -- 172.21.15.46:0/2166567941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4102d40 msgr2=0x7fb9c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 --2- 172.21.15.46:0/2166567941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4102d40 0x7fb9c4103120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009900 tx=0x7fb9b402a480 comp rx=0 tx=0).stop 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 -- 172.21.15.46:0/2166567941 shutdown_connections 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 --2- 172.21.15.46:0/2166567941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c410c170 0x7fb9c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 --2- 172.21.15.46:0/2166567941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4103660 0x7fb9c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 --2- 172.21.15.46:0/2166567941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c4102d40 0x7fb9c4103120 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 -- 172.21.15.46:0/2166567941 >> 172.21.15.46:0/2166567941 conn(0x7fb9c40fc7a0 msgr2=0x7fb9c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 -- 172.21.15.46:0/2166567941 shutdown_connections 2024-09-12T02:48:30.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.875+0000 7fb9c887c640 1 -- 172.21.15.46:0/2166567941 wait complete. 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 Processor -- start 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 -- start start 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102d40 0x7fb9c4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4103660 0x7fb9c41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111e20 con 0x7fb9c410c170 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111ca0 con 0x7fb9c4103660 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c887c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4111fa0 con 0x7fb9c4102d40 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48436/0 (socket says 172.21.15.46:48436) 2024-09-12T02:48:30.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 -- 172.21.15.46:0/1728887971 learned_addr learned my addr 172.21.15.46:0/1728887971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c259c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102d40 0x7fb9c4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 -- 172.21.15.46:0/1728887971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102d40 msgr2=0x7fb9c4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c1d9b640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4103660 0x7fb9c41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102d40 0x7fb9c4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 -- 172.21.15.46:0/1728887971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4103660 msgr2=0x7fb9c41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4103660 0x7fb9c41988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.876+0000 7fb9c2d9d640 1 -- 172.21.15.46:0/1728887971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419d330 con 0x7fb9c410c170 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9c2d9d640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00eaa0 tx=0x7fb9bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00cdb0 con 0x7fb9c410c170 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419d620 con 0x7fb9c410c170 2024-09-12T02:48:30.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc005020 con 0x7fb9c410c170 2024-09-12T02:48:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0116d0 con 0x7fb9c410c170 2024-09-12T02:48:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.877+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41a4f00 con 0x7fb9c410c170 2024-09-12T02:48:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.878+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c4103120 con 0x7fb9c410c170 2024-09-12T02:48:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.879+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb9bc0051d0 con 0x7fb9c410c170 2024-09-12T02:48:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.880+0000 7fb9b37fe640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 0x7fb9ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.880+0000 7fb9c259c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 0x7fb9ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.880+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb9bc0cf5c0 con 0x7fb9c410c170 2024-09-12T02:48:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.881+0000 7fb9c259c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 0x7fb9ac0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4031440 tx=0x7fb9b4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:30.885+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9bc016030 con 0x7fb9c410c170 2024-09-12T02:48:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.088+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb9c4062630 con 0x7fb9ac0a2c10 2024-09-12T02:48:31.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.094+0000 7fb9b37fe640 1 -- 172.21.15.46:0/1728887971 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb9c4062630 con 0x7fb9ac0a2c10 2024-09-12T02:48:31.095 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/6566f6e7-a2f5-4055-9c2d-0504e61c8664 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.096+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 msgr2=0x7fb9ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.096+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 0x7fb9ac0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4031440 tx=0x7fb9b4005b50 comp rx=0 tx=0).stop 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.096+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 msgr2=0x7fb9c419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.096+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00eaa0 tx=0x7fb9bc00ef60 comp rx=0 tx=0).stop 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 shutdown_connections 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c410c170 0x7fb9c419cc50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:31.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9c4103660 0x7fb9c41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fb9ac0a2c10 0x7fb9ac0a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 --2- 172.21.15.46:0/1728887971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4102d40 0x7fb9c4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 >> 172.21.15.46:0/1728887971 conn(0x7fb9c40fc7a0 msgr2=0x7fb9c40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 shutdown_connections 2024-09-12T02:48:31.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:31.097+0000 7fb9c887c640 1 -- 172.21.15.46:0/1728887971 wait complete. 2024-09-12T02:48:31.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:31 smithi046 ceph-mon[30951]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:31.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:31 smithi046 ceph-mon[30951]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:31 smithi145 ceph-mon[34674]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:31 smithi145 ceph-mon[34674]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:31 smithi196 ceph-mon[34587]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T02:48:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:31 smithi196 ceph-mon[34587]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:48:31.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:48:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 -- 172.21.15.46:0/875677517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 msgr2=0x7f445010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 --2- 172.21.15.46:0/875677517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445010ea90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4448009ca0 tx=0x7f444802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 -- 172.21.15.46:0/875677517 shutdown_connections 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 --2- 172.21.15.46:0/875677517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445010ea90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 --2- 172.21.15.46:0/875677517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f44501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 --2- 172.21.15.46:0/875677517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 0x7f44501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 -- 172.21.15.46:0/875677517 >> 172.21.15.46:0/875677517 conn(0x7f44500fd8c0 msgr2=0x7f44500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 -- 172.21.15.46:0/875677517 shutdown_connections 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.679+0000 7f4454c7e640 1 -- 172.21.15.46:0/875677517 wait complete. 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 Processor -- start 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 -- start start 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 0x7f44501981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f4450198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111e20 con 0x7f4450106f10 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111ca0 con 0x7f4450101de0 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.680+0000 7f4454c7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111fa0 con 0x7f4450102790 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48454/0 (socket says 172.21.15.46:48454) 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 -- 172.21.15.46:0/2142622044 learned_addr learned my addr 172.21.15.46:0/2142622044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444e59c640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 0x7f44501981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444dd9b640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f4450198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 -- 172.21.15.46:0/2142622044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 msgr2=0x7f4450198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f4450198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 -- 172.21.15.46:0/2142622044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 msgr2=0x7f44501981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 0x7f44501981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 -- 172.21.15.46:0/2142622044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445019d170 con 0x7f4450106f10 2024-09-12T02:48:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444dd9b640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f4450198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f444ed9d640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4448009dc0 tx=0x7f4448003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448004040 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4448004690 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445019d400 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448030c90 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.681+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a4d40 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.683+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44501021c0 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.684+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44480041e0 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.684+0000 7f443f7fe640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 0x7f44380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.685+0000 7f444e59c640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 0x7f44380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.685+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f44480f0b50 con 0x7f4450106f10 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.685+0000 7f444e59c640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 0x7f44380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f444000abe0 tx=0x7f444000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.688+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44480bd7c0 con 0x7f4450106f10 2024-09-12T02:48:32.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.902+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f445010a150 con 0x7f4450106f10 2024-09-12T02:48:32.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.903+0000 7f443f7fe640 1 -- 172.21.15.46:0/2142622044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7f44480bd0c0 con 0x7f4450106f10 2024-09-12T02:48:32.904 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:32.904 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:48:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 msgr2=0x7f44380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 0x7f44380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f444000abe0 tx=0x7f444000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 msgr2=0x7f445019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4448009dc0 tx=0x7f4448003ba0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 shutdown_connections 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450106f10 0x7f445019ca90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f44380a2c10 0x7f44380a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f4450198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 --2- 172.21.15.46:0/2142622044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450101de0 0x7f44501981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.905+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 >> 172.21.15.46:0/2142622044 conn(0x7f44500fd8c0 msgr2=0x7f44500feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.906+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 shutdown_connections 2024-09-12T02:48:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:32.906+0000 7f4454c7e640 1 -- 172.21.15.46:0/2142622044 wait complete. 2024-09-12T02:48:33.594 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:33 smithi046 ceph-mon[30951]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T02:48:33.594 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:33 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2142622044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:48:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:33 smithi145 ceph-mon[34674]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T02:48:33.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:33 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2142622044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:48:33.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:48:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:33 smithi196 ceph-mon[34587]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T02:48:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:33 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2142622044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:48:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 -- 172.21.15.46:0/2014868380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 msgr2=0x7f88ec106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 --2- 172.21.15.46:0/2014868380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec106e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009930 tx=0x7f88dc02a420 comp rx=0 tx=0).stop 2024-09-12T02:48:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 -- 172.21.15.46:0/2014868380 shutdown_connections 2024-09-12T02:48:34.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 --2- 172.21.15.46:0/2014868380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 0x7f88ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 --2- 172.21.15.46:0/2014868380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 --2- 172.21.15.46:0/2014868380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec106e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 -- 172.21.15.46:0/2014868380 >> 172.21.15.46:0/2014868380 conn(0x7f88ec0fc7a0 msgr2=0x7f88ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 -- 172.21.15.46:0/2014868380 shutdown_connections 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.414+0000 7f88f1acc640 1 -- 172.21.15.46:0/2014868380 wait complete. 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 Processor -- start 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 -- start start 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 0x7f88ec199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:34.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec110d30 con 0x7f88ec106a80 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec110bb0 con 0x7f88ec100a80 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88f1acc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec110eb0 con 0x7f88ec101420 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52392/0 (socket says 172.21.15.46:52392) 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.415+0000 7f88ebfff640 1 -- 172.21.15.46:0/346276133 learned_addr learned my addr 172.21.15.46:0/346276133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88eaffd640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 0x7f88ec199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 -- 172.21.15.46:0/346276133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 msgr2=0x7f88ec199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 0x7f88ec199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 -- 172.21.15.46:0/346276133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 msgr2=0x7f88ec1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 -- 172.21.15.46:0/346276133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1a4790 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88eb7fe640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88ebfff640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88e400eaa0 tx=0x7f88e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e400cdb0 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e4005020 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1a4960 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e40116d0 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.416+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a4e70 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.417+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec100ee0 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.419+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f88e40051d0 con 0x7f88ec106a80 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.419+0000 7f88e8ff9640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 0x7f88d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:34.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.419+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f88e40cf910 con 0x7f88ec106a80 2024-09-12T02:48:34.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.419+0000 7f88eb7fe640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 0x7f88d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:34.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.420+0000 7f88eb7fe640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 0x7f88d40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:34.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.423+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88e4016030 con 0x7f88ec106a80 2024-09-12T02:48:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.598+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88ec06ad80 con 0x7f88ec106a80 2024-09-12T02:48:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.599+0000 7f88e8ff9640 1 -- 172.21.15.46:0/346276133 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24188 (secure 0 0 0) 0x7f88ec06ad80 con 0x7f88ec106a80 2024-09-12T02:48:34.600 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:34.600 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:47:50.856239+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:48:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.601+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 msgr2=0x7f88d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.601+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 0x7f88d40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.601+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 msgr2=0x7f88ec1a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.601+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88e400eaa0 tx=0x7f88e400ef60 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 shutdown_connections 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ec106a80 0x7f88ec1a4250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88d40a2c10 0x7f88d40a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ec101420 0x7f88ec199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 --2- 172.21.15.46:0/346276133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88ec100a80 0x7f88ec1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 >> 172.21.15.46:0/346276133 conn(0x7f88ec0fc7a0 msgr2=0x7f88ec0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 shutdown_connections 2024-09-12T02:48:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:34.602+0000 7f88f1acc640 1 -- 172.21.15.46:0/346276133 wait complete. 2024-09-12T02:48:35.260 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:48:35.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:48:35.521 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:35.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:35 smithi046 ceph-mon[30951]: pgmap v317: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:35.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:35 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/346276133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:35.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:35 smithi145 ceph-mon[34674]: pgmap v317: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:35.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:35 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/346276133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:35.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:35 smithi196 ceph-mon[34587]: pgmap v317: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:35.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:35 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/346276133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:36.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 -- 172.21.15.46:0/139641896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70106f10 msgr2=0x7f7a7010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:36.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/139641896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70106f10 0x7f7a7010ea90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009ca0 tx=0x7f7a6802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 -- 172.21.15.46:0/139641896 shutdown_connections 2024-09-12T02:48:36.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/139641896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70106f10 0x7f7a7010ea90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/139641896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 0x7f7a701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/139641896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70101de0 0x7f7a701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 -- 172.21.15.46:0/139641896 >> 172.21.15.46:0/139641896 conn(0x7f7a700fd8c0 msgr2=0x7f7a700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 -- 172.21.15.46:0/139641896 shutdown_connections 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 -- 172.21.15.46:0/139641896 wait complete. 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.013+0000 7f7a77c7f640 1 Processor -- start 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 -- start start 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 0x7f7a70198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 0x7f7a7019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70111e20 con 0x7f7a70101de0 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70111ca0 con 0x7f7a70106f10 2024-09-12T02:48:36.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a77c7f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70111fa0 con 0x7f7a70102790 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52408/0 (socket says 172.21.15.46:52408) 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 -- 172.21.15.46:0/2139186062 learned_addr learned my addr 172.21.15.46:0/2139186062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a7621c640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 0x7f7a7019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a7521a640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 0x7f7a70198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 -- 172.21.15.46:0/2139186062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 msgr2=0x7f7a70198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 0x7f7a70198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 -- 172.21.15.46:0/2139186062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 msgr2=0x7f7a7019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 0x7f7a7019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 -- 172.21.15.46:0/2139186062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7019d2f0 con 0x7f7a70101de0 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a7621c640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 0x7f7a7019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.014+0000 7f7a75a1b640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:36.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.015+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c017070 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.015+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7019d5e0 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.015+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6c009c30 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.015+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00f620 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.015+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a4ec0 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.016+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a701021c0 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.017+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a6c0109a0 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.018+0000 7f7a62ffd640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 0x7f7a5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.018+0000 7f7a7521a640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 0x7f7a5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.018+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f7a6c0cf170 con 0x7f7a70101de0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.018+0000 7f7a7521a640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 0x7f7a5c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005fd0 tx=0x7f7a6400b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:36.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.021+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a6c009de0 con 0x7f7a70101de0 2024-09-12T02:48:36.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.196+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a70062630 con 0x7f7a5c0a2c10 2024-09-12T02:48:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.205+0000 7f7a62ffd640 1 -- 172.21.15.46:0/2139186062 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340562 (secure 0 0 0) 0x7f7a70062630 con 0x7f7a5c0a2c10 2024-09-12T02:48:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:36.212 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:48:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 msgr2=0x7f7a5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 0x7f7a5c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005fd0 tx=0x7f7a6400b510 comp rx=0 tx=0).stop 2024-09-12T02:48:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 msgr2=0x7f7a70198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00de10 comp rx=0 tx=0).stop 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 shutdown_connections 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7a70106f10 0x7f7a7019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7a5c0a2c10 0x7f7a5c0a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a70102790 0x7f7a70198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 --2- 172.21.15.46:0/2139186062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a70101de0 0x7f7a70198340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 >> 172.21.15.46:0/2139186062 conn(0x7f7a700fd8c0 msgr2=0x7f7a700fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 shutdown_connections 2024-09-12T02:48:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:36.209+0000 7f7a77c7f640 1 -- 172.21.15.46:0/2139186062 wait complete. 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T02:48:36.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:36 smithi046 ceph-mon[30951]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T02:48:36.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:36 smithi145 ceph-mon[34674]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T02:48:36.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-12T02:48:36.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:36 smithi196 ceph-mon[34587]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:36.949 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-12T02:48:34.272193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":232,"num_read_kb":217,"num_write":336,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369232,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572592,"statfs":{"total":1151940427776,"available":1151562334208,"internally_reserved":0,"allocated":11943936,"data_stored":7672349,"data_compressed":319986,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40128,"num_objects":6,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":73,"num_write":34,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003635"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.235863+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:19.235863+0000","last_peered":"2024-09-12T02:48:19.235863+0000","last_clean":"2024-09-12T02:48:19.235863+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:19.235863+0000","last_undegraded":"2024-09-12T02:48:19.235863+0000","last_fullsized":"2024-09-12T02:48:19.235863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281137+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:47:52.281137+0000","last_peered":"2024-09-12T02:47:52.281137+0000","last_clean":"2024-09-12T02:47:52.281137+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:47:52.281137+0000","last_undegraded":"2024-09-12T02:47:52.281137+0000","last_fullsized":"2024-09-12T02:47:52.281137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403462+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:47:52.403462+0000","last_peered":"2024-09-12T02:47:52.403462+0000","last_clean":"2024-09-12T02:47:52.403462+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:47:52.403462+0000","last_undegraded":"2024-09-12T02:47:52.403462+0000","last_fullsized":"2024-09-12T02:47:52.403462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281434+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:47:52.281434+0000","last_peered":"2024-09-12T02:47:52.281434+0000","last_clean":"2024-09-12T02:47:52.281434+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:47:52.281434+0000","last_undegraded":"2024-09-12T02:47:52.281434+0000","last_fullsized":"2024-09-12T02:47:52.281434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403846+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:47:52.403846+0000","last_peered":"2024-09-12T02:47:52.403846+0000","last_clean":"2024-09-12T02:47:52.403846+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:47:52.403846+0000","last_undegraded":"2024-09-12T02:47:52.403846+0000","last_fullsized":"2024-09-12T02:47:52.403846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404200+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:47:52.404200+0000","last_peered":"2024-09-12T02:47:52.404200+0000","last_clean":"2024-09-12T02:47:52.404200+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:47:52.404200+0000","last_undegraded":"2024-09-12T02:47:52.404200+0000","last_fullsized":"2024-09-12T02:47:52.404200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234397+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:19.234397+0000","last_peered":"2024-09-12T02:48:19.234397+0000","last_clean":"2024-09-12T02:48:19.234397+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:19.234397+0000","last_undegraded":"2024-09-12T02:48:19.234397+0000","last_fullsized":"2024-09-12T02:48:19.234397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404446+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:47:52.404446+0000","last_peered":"2024-09-12T02:47:52.404446+0000","last_clean":"2024-09-12T02:47:52.404446+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:47:52.404446+0000","last_undegraded":"2024-09-12T02:47:52.404446+0000","last_fullsized":"2024-09-12T02:47:52.404446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405523+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:47:52.405523+0000","last_peered":"2024-09-12T02:47:52.405523+0000","last_clean":"2024-09-12T02:47:52.405523+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:47:52.405523+0000","last_undegraded":"2024-09-12T02:47:52.405523+0000","last_fullsized":"2024-09-12T02:47:52.405523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404295+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:47:52.404295+0000","last_peered":"2024-09-12T02:47:52.404295+0000","last_clean":"2024-09-12T02:47:52.404295+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:47:52.404295+0000","last_undegraded":"2024-09-12T02:47:52.404295+0000","last_fullsized":"2024-09-12T02:47:52.404295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226609+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:47:52.226609+0000","last_peered":"2024-09-12T02:47:52.226609+0000","last_clean":"2024-09-12T02:47:52.226609+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:47:52.226609+0000","last_undegraded":"2024-09-12T02:47:52.226609+0000","last_fullsized":"2024-09-12T02:47:52.226609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404157+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:47:52.404157+0000","last_peered":"2024-09-12T02:47:52.404157+0000","last_clean":"2024-09-12T02:47:52.404157+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:47:52.404157+0000","last_undegraded":"2024-09-12T02:47:52.404157+0000","last_fullsized":"2024-09-12T02:47:52.404157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405358+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:47:52.405358+0000","last_peered":"2024-09-12T02:47:52.405358+0000","last_clean":"2024-09-12T02:47:52.405358+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:47:52.405358+0000","last_undegraded":"2024-09-12T02:47:52.405358+0000","last_fullsized":"2024-09-12T02:47:52.405358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.222528+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:18.222528+0000","last_peered":"2024-09-12T02:48:18.222528+0000","last_clean":"2024-09-12T02:48:18.222528+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:18.222528+0000","last_undegraded":"2024-09-12T02:48:18.222528+0000","last_fullsized":"2024-09-12T02:48:18.222528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405095+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:47:52.405095+0000","last_peered":"2024-09-12T02:47:52.405095+0000","last_clean":"2024-09-12T02:47:52.405095+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:47:52.405095+0000","last_undegraded":"2024-09-12T02:47:52.405095+0000","last_fullsized":"2024-09-12T02:47:52.405095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280980+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:47:52.280980+0000","last_peered":"2024-09-12T02:47:52.280980+0000","last_clean":"2024-09-12T02:47:52.280980+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:47:52.280980+0000","last_undegraded":"2024-09-12T02:47:52.280980+0000","last_fullsized":"2024-09-12T02:47:52.280980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403643+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:47:52.403643+0000","last_peered":"2024-09-12T02:47:52.403643+0000","last_clean":"2024-09-12T02:47:52.403643+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:47:52.403643+0000","last_undegraded":"2024-09-12T02:47:52.403643+0000","last_fullsized":"2024-09-12T02:47:52.403643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405281+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:47:52.405281+0000","last_peered":"2024-09-12T02:47:52.405281+0000","last_clean":"2024-09-12T02:47:52.405281+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:47:52.405281+0000","last_undegraded":"2024-09-12T02:47:52.405281+0000","last_fullsized":"2024-09-12T02:47:52.405281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141492+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:47:51.141492+0000","last_peered":"2024-09-12T02:47:51.141492+0000","last_clean":"2024-09-12T02:47:51.141492+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:47:51.141492+0000","last_undegraded":"2024-09-12T02:47:51.141492+0000","last_fullsized":"2024-09-12T02:47:51.141492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223309+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:18.223309+0000","last_peered":"2024-09-12T02:48:18.223309+0000","last_clean":"2024-09-12T02:48:18.223309+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:18.223309+0000","last_undegraded":"2024-09-12T02:48:18.223309+0000","last_fullsized":"2024-09-12T02:48:18.223309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141859+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:47:51.141859+0000","last_peered":"2024-09-12T02:47:51.141859+0000","last_clean":"2024-09-12T02:47:51.141859+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:47:51.141859+0000","last_undegraded":"2024-09-12T02:47:51.141859+0000","last_fullsized":"2024-09-12T02:47:51.141859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403682+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:47:52.403682+0000","last_peered":"2024-09-12T02:47:52.403682+0000","last_clean":"2024-09-12T02:47:52.403682+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:47:52.403682+0000","last_undegraded":"2024-09-12T02:47:52.403682+0000","last_fullsized":"2024-09-12T02:47:52.403682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223316+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:18.223316+0000","last_peered":"2024-09-12T02:48:18.223316+0000","last_clean":"2024-09-12T02:48:18.223316+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:18.223316+0000","last_undegraded":"2024-09-12T02:48:18.223316+0000","last_fullsized":"2024-09-12T02:48:18.223316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404343+0000","last_change":"2024-09-12T02:47:48.159271+0000","last_active":"2024-09-12T02:47:52.404343+0000","last_peered":"2024-09-12T02:47:52.404343+0000","last_clean":"2024-09-12T02:47:52.404343+0000","last_became_active":"2024-09-12T02:47:48.158833+0000","last_became_peered":"2024-09-12T02:47:48.158833+0000","last_unstale":"2024-09-12T02:47:52.404343+0000","last_undegraded":"2024-09-12T02:47:52.404343+0000","last_fullsized":"2024-09-12T02:47:52.404343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:57.861400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403831+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:47:52.403831+0000","last_peered":"2024-09-12T02:47:52.403831+0000","last_clean":"2024-09-12T02:47:52.403831+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:47:52.403831+0000","last_undegraded":"2024-09-12T02:47:52.403831+0000","last_fullsized":"2024-09-12T02:47:52.403831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223107+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:18.223107+0000","last_peered":"2024-09-12T02:48:18.223107+0000","last_clean":"2024-09-12T02:48:18.223107+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:18.223107+0000","last_undegraded":"2024-09-12T02:48:18.223107+0000","last_fullsized":"2024-09-12T02:48:18.223107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403534+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:47:52.403534+0000","last_peered":"2024-09-12T02:47:52.403534+0000","last_clean":"2024-09-12T02:47:52.403534+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:47:52.403534+0000","last_undegraded":"2024-09-12T02:47:52.403534+0000","last_fullsized":"2024-09-12T02:47:52.403534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281499+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:47:52.281499+0000","last_peered":"2024-09-12T02:47:52.281499+0000","last_clean":"2024-09-12T02:47:52.281499+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:47:52.281499+0000","last_undegraded":"2024-09-12T02:47:52.281499+0000","last_fullsized":"2024-09-12T02:47:52.281499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:30.114832+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:30.114832+0000","last_peered":"2024-09-12T02:48:30.114832+0000","last_clean":"2024-09-12T02:48:30.114832+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:30.114832+0000","last_undegraded":"2024-09-12T02:48:30.114832+0000","last_fullsized":"2024-09-12T02:48:30.114832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280833+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:47:52.280833+0000","last_peered":"2024-09-12T02:47:52.280833+0000","last_clean":"2024-09-12T02:47:52.280833+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:47:52.280833+0000","last_undegraded":"2024-09-12T02:47:52.280833+0000","last_fullsized":"2024-09-12T02:47:52.280833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499352+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:47:52.499352+0000","last_peered":"2024-09-12T02:47:52.499352+0000","last_clean":"2024-09-12T02:47:52.499352+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:47:52.499352+0000","last_undegraded":"2024-09-12T02:47:52.499352+0000","last_fullsized":"2024-09-12T02:47:52.499352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.220889+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:18.220889+0000","last_peered":"2024-09-12T02:48:18.220889+0000","last_clean":"2024-09-12T02:48:18.220889+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:18.220889+0000","last_undegraded":"2024-09-12T02:48:18.220889+0000","last_fullsized":"2024-09-12T02:48:18.220889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:47:51.141552+0000","last_peered":"2024-09-12T02:47:51.141552+0000","last_clean":"2024-09-12T02:47:51.141552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:47:51.141552+0000","last_undegraded":"2024-09-12T02:47:51.141552+0000","last_fullsized":"2024-09-12T02:47:51.141552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141747+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:47:51.141747+0000","last_peered":"2024-09-12T02:47:51.141747+0000","last_clean":"2024-09-12T02:47:51.141747+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:47:51.141747+0000","last_undegraded":"2024-09-12T02:47:51.141747+0000","last_fullsized":"2024-09-12T02:47:51.141747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141700+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:47:51.141700+0000","last_peered":"2024-09-12T02:47:51.141700+0000","last_clean":"2024-09-12T02:47:51.141700+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:47:51.141700+0000","last_undegraded":"2024-09-12T02:47:51.141700+0000","last_fullsized":"2024-09-12T02:47:51.141700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403573+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:47:52.403573+0000","last_peered":"2024-09-12T02:47:52.403573+0000","last_clean":"2024-09-12T02:47:52.403573+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:47:52.403573+0000","last_undegraded":"2024-09-12T02:47:52.403573+0000","last_fullsized":"2024-09-12T02:47:52.403573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141682+0000","last_change":"2024-09-12T02:47:41.115955+0000","last_active":"2024-09-12T02:47:51.141682+0000","last_peered":"2024-09-12T02:47:51.141682+0000","last_clean":"2024-09-12T02:47:51.141682+0000","last_became_active":"2024-09-12T02:47:41.115604+0000","last_became_peered":"2024-09-12T02:47:41.115604+0000","last_unstale":"2024-09-12T02:47:51.141682+0000","last_undegraded":"2024-09-12T02:47:51.141682+0000","last_fullsized":"2024-09-12T02:47:51.141682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:46.153312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405554+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:47:52.405554+0000","last_peered":"2024-09-12T02:47:52.405554+0000","last_clean":"2024-09-12T02:47:52.405554+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:47:52.405554+0000","last_undegraded":"2024-09-12T02:47:52.405554+0000","last_fullsized":"2024-09-12T02:47:52.405554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499174+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:47:52.499174+0000","last_peered":"2024-09-12T02:47:52.499174+0000","last_clean":"2024-09-12T02:47:52.499174+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:47:52.499174+0000","last_undegraded":"2024-09-12T02:47:52.499174+0000","last_fullsized":"2024-09-12T02:47:52.499174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226339+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:47:52.226339+0000","last_peered":"2024-09-12T02:47:52.226339+0000","last_clean":"2024-09-12T02:47:52.226339+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:47:52.226339+0000","last_undegraded":"2024-09-12T02:47:52.226339+0000","last_fullsized":"2024-09-12T02:47:52.226339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:28.118010+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:48:28.118010+0000","last_peered":"2024-09-12T02:48:28.118010+0000","last_clean":"2024-09-12T02:48:28.118010+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:48:28.118010+0000","last_undegraded":"2024-09-12T02:48:28.118010+0000","last_fullsized":"2024-09-12T02:48:28.118010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499186+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:47:52.499186+0000","last_peered":"2024-09-12T02:47:52.499186+0000","last_clean":"2024-09-12T02:47:52.499186+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:47:52.499186+0000","last_undegraded":"2024-09-12T02:47:52.499186+0000","last_fullsized":"2024-09-12T02:47:52.499186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.498800+0000","last_change":"2024-09-12T02:47:41.114262+0000","last_active":"2024-09-12T02:47:52.498800+0000","last_peered":"2024-09-12T02:47:52.498800+0000","last_clean":"2024-09-12T02:47:52.498800+0000","last_became_active":"2024-09-12T02:47:41.113951+0000","last_became_peered":"2024-09-12T02:47:41.113951+0000","last_unstale":"2024-09-12T02:47:52.498800+0000","last_undegraded":"2024-09-12T02:47:52.498800+0000","last_fullsized":"2024-09-12T02:47:52.498800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:38.948691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.233809+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:19.233809+0000","last_peered":"2024-09-12T02:48:19.233809+0000","last_clean":"2024-09-12T02:48:19.233809+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:19.233809+0000","last_undegraded":"2024-09-12T02:48:19.233809+0000","last_fullsized":"2024-09-12T02:48:19.233809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226749+0000","last_change":"2024-09-12T02:47:48.157776+0000","last_active":"2024-09-12T02:47:52.226749+0000","last_peered":"2024-09-12T02:47:52.226749+0000","last_clean":"2024-09-12T02:47:52.226749+0000","last_became_active":"2024-09-12T02:47:48.157488+0000","last_became_peered":"2024-09-12T02:47:48.157488+0000","last_unstale":"2024-09-12T02:47:52.226749+0000","last_undegraded":"2024-09-12T02:47:52.226749+0000","last_fullsized":"2024-09-12T02:47:52.226749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:32.011455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281134+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:47:52.281134+0000","last_peered":"2024-09-12T02:47:52.281134+0000","last_clean":"2024-09-12T02:47:52.281134+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:47:52.281134+0000","last_undegraded":"2024-09-12T02:47:52.281134+0000","last_fullsized":"2024-09-12T02:47:52.281134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:20.103149+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:20.103149+0000","last_peered":"2024-09-12T02:48:20.103149+0000","last_clean":"2024-09-12T02:48:20.103149+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:20.103149+0000","last_undegraded":"2024-09-12T02:48:20.103149+0000","last_fullsized":"2024-09-12T02:48:20.103149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405158+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:47:52.405158+0000","last_peered":"2024-09-12T02:47:52.405158+0000","last_clean":"2024-09-12T02:47:52.405158+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:47:52.405158+0000","last_undegraded":"2024-09-12T02:47:52.405158+0000","last_fullsized":"2024-09-12T02:47:52.405158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405866+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:47:52.405866+0000","last_peered":"2024-09-12T02:47:52.405866+0000","last_clean":"2024-09-12T02:47:52.405866+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:47:52.405866+0000","last_undegraded":"2024-09-12T02:47:52.405866+0000","last_fullsized":"2024-09-12T02:47:52.405866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.406006+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:47:52.406006+0000","last_peered":"2024-09-12T02:47:52.406006+0000","last_clean":"2024-09-12T02:47:52.406006+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:47:52.406006+0000","last_undegraded":"2024-09-12T02:47:52.406006+0000","last_fullsized":"2024-09-12T02:47:52.406006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.235709+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:19.235709+0000","last_peered":"2024-09-12T02:48:19.235709+0000","last_clean":"2024-09-12T02:48:19.235709+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:19.235709+0000","last_undegraded":"2024-09-12T02:48:19.235709+0000","last_fullsized":"2024-09-12T02:48:19.235709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404343+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:47:52.404343+0000","last_peered":"2024-09-12T02:47:52.404343+0000","last_clean":"2024-09-12T02:47:52.404343+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:47:52.404343+0000","last_undegraded":"2024-09-12T02:47:52.404343+0000","last_fullsized":"2024-09-12T02:47:52.404343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405711+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:47:52.405711+0000","last_peered":"2024-09-12T02:47:52.405711+0000","last_clean":"2024-09-12T02:47:52.405711+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:47:52.405711+0000","last_undegraded":"2024-09-12T02:47:52.405711+0000","last_fullsized":"2024-09-12T02:47:52.405711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226827+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:47:52.226827+0000","last_peered":"2024-09-12T02:47:52.226827+0000","last_clean":"2024-09-12T02:47:52.226827+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:47:52.226827+0000","last_undegraded":"2024-09-12T02:47:52.226827+0000","last_fullsized":"2024-09-12T02:47:52.226827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403541+0000","last_change":"2024-09-12T02:47:48.153637+0000","last_active":"2024-09-12T02:47:52.403541+0000","last_peered":"2024-09-12T02:47:52.403541+0000","last_clean":"2024-09-12T02:47:52.403541+0000","last_became_active":"2024-09-12T02:47:48.153180+0000","last_became_peered":"2024-09-12T02:47:48.153180+0000","last_unstale":"2024-09-12T02:47:52.403541+0000","last_undegraded":"2024-09-12T02:47:52.403541+0000","last_fullsized":"2024-09-12T02:47:52.403541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:59.876118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405875+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:47:52.405875+0000","last_peered":"2024-09-12T02:47:52.405875+0000","last_clean":"2024-09-12T02:47:52.405875+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:47:52.405875+0000","last_undegraded":"2024-09-12T02:47:52.405875+0000","last_fullsized":"2024-09-12T02:47:52.405875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499252+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:47:52.499252+0000","last_peered":"2024-09-12T02:47:52.499252+0000","last_clean":"2024-09-12T02:47:52.499252+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:47:52.499252+0000","last_undegraded":"2024-09-12T02:47:52.499252+0000","last_fullsized":"2024-09-12T02:47:52.499252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226234+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:47:52.226234+0000","last_peered":"2024-09-12T02:47:52.226234+0000","last_clean":"2024-09-12T02:47:52.226234+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:47:52.226234+0000","last_undegraded":"2024-09-12T02:47:52.226234+0000","last_fullsized":"2024-09-12T02:47:52.226234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141930+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:47:51.141930+0000","last_peered":"2024-09-12T02:47:51.141930+0000","last_clean":"2024-09-12T02:47:51.141930+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:47:51.141930+0000","last_undegraded":"2024-09-12T02:47:51.141930+0000","last_fullsized":"2024-09-12T02:47:51.141930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:20.104615+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:20.104615+0000","last_peered":"2024-09-12T02:48:20.104615+0000","last_clean":"2024-09-12T02:48:20.104615+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:20.104615+0000","last_undegraded":"2024-09-12T02:48:20.104615+0000","last_fullsized":"2024-09-12T02:48:20.104615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226286+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:47:52.226286+0000","last_peered":"2024-09-12T02:47:52.226286+0000","last_clean":"2024-09-12T02:47:52.226286+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:47:52.226286+0000","last_undegraded":"2024-09-12T02:47:52.226286+0000","last_fullsized":"2024-09-12T02:47:52.226286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226303+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:47:52.226303+0000","last_peered":"2024-09-12T02:47:52.226303+0000","last_clean":"2024-09-12T02:47:52.226303+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:47:52.226303+0000","last_undegraded":"2024-09-12T02:47:52.226303+0000","last_fullsized":"2024-09-12T02:47:52.226303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.236279+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:19.236279+0000","last_peered":"2024-09-12T02:48:19.236279+0000","last_clean":"2024-09-12T02:48:19.236279+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:19.236279+0000","last_undegraded":"2024-09-12T02:48:19.236279+0000","last_fullsized":"2024-09-12T02:48:19.236279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499277+0000","last_change":"2024-09-12T02:47:48.149728+0000","last_active":"2024-09-12T02:47:52.499277+0000","last_peered":"2024-09-12T02:47:52.499277+0000","last_clean":"2024-09-12T02:47:52.499277+0000","last_became_active":"2024-09-12T02:47:48.149314+0000","last_became_peered":"2024-09-12T02:47:48.149314+0000","last_unstale":"2024-09-12T02:47:52.499277+0000","last_undegraded":"2024-09-12T02:47:52.499277+0000","last_fullsized":"2024-09-12T02:47:52.499277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:34.293981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404831+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:47:52.404831+0000","last_peered":"2024-09-12T02:47:52.404831+0000","last_clean":"2024-09-12T02:47:52.404831+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:47:52.404831+0000","last_undegraded":"2024-09-12T02:47:52.404831+0000","last_fullsized":"2024-09-12T02:47:52.404831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:30.121214+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:48:30.121214+0000","last_peered":"2024-09-12T02:48:30.121214+0000","last_clean":"2024-09-12T02:48:30.121214+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:48:30.121214+0000","last_undegraded":"2024-09-12T02:48:30.121214+0000","last_fullsized":"2024-09-12T02:48:30.121214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.142013+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:47:51.142013+0000","last_peered":"2024-09-12T02:47:51.142013+0000","last_clean":"2024-09-12T02:47:51.142013+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:47:51.142013+0000","last_undegraded":"2024-09-12T02:47:51.142013+0000","last_fullsized":"2024-09-12T02:47:51.142013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405675+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:47:52.405675+0000","last_peered":"2024-09-12T02:47:52.405675+0000","last_clean":"2024-09-12T02:47:52.405675+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:47:52.405675+0000","last_undegraded":"2024-09-12T02:47:52.405675+0000","last_fullsized":"2024-09-12T02:47:52.405675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.222938+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:18.222938+0000","last_peered":"2024-09-12T02:48:18.222938+0000","last_clean":"2024-09-12T02:48:18.222938+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:18.222938+0000","last_undegraded":"2024-09-12T02:48:18.222938+0000","last_fullsized":"2024-09-12T02:48:18.222938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141558+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:47:51.141558+0000","last_peered":"2024-09-12T02:47:51.141558+0000","last_clean":"2024-09-12T02:47:51.141558+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:47:51.141558+0000","last_undegraded":"2024-09-12T02:47:51.141558+0000","last_fullsized":"2024-09-12T02:47:51.141558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.498955+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:47:52.498955+0000","last_peered":"2024-09-12T02:47:52.498955+0000","last_clean":"2024-09-12T02:47:52.498955+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:47:52.498955+0000","last_undegraded":"2024-09-12T02:47:52.498955+0000","last_fullsized":"2024-09-12T02:47:52.498955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234907+0000","last_change":"2024-09-12T02:47:39.062615+0000","last_active":"2024-09-12T02:48:19.234907+0000","last_peered":"2024-09-12T02:48:19.234907+0000","last_clean":"2024-09-12T02:48:19.234907+0000","last_became_active":"2024-09-12T02:47:39.062174+0000","last_became_peered":"2024-09-12T02:47:39.062174+0000","last_unstale":"2024-09-12T02:48:19.234907+0000","last_undegraded":"2024-09-12T02:48:19.234907+0000","last_fullsized":"2024-09-12T02:48:19.234907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:10.443649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226668+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:47:52.226668+0000","last_peered":"2024-09-12T02:47:52.226668+0000","last_clean":"2024-09-12T02:47:52.226668+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:47:52.226668+0000","last_undegraded":"2024-09-12T02:47:52.226668+0000","last_fullsized":"2024-09-12T02:47:52.226668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404222+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:47:52.404222+0000","last_peered":"2024-09-12T02:47:52.404222+0000","last_clean":"2024-09-12T02:47:52.404222+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:47:52.404222+0000","last_undegraded":"2024-09-12T02:47:52.404222+0000","last_fullsized":"2024-09-12T02:47:52.404222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'25","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:31.125929+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:48:31.125929+0000","last_peered":"2024-09-12T02:48:31.125929+0000","last_clean":"2024-09-12T02:48:31.125929+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:48:31.125929+0000","last_undegraded":"2024-09-12T02:48:31.125929+0000","last_fullsized":"2024-09-12T02:48:31.125929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":57,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141327+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:47:51.141327+0000","last_peered":"2024-09-12T02:47:51.141327+0000","last_clean":"2024-09-12T02:47:51.141327+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:47:51.141327+0000","last_undegraded":"2024-09-12T02:47:51.141327+0000","last_fullsized":"2024-09-12T02:47:51.141327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404520+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:47:52.404520+0000","last_peered":"2024-09-12T02:47:52.404520+0000","last_clean":"2024-09-12T02:47:52.404520+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:47:52.404520+0000","last_undegraded":"2024-09-12T02:47:52.404520+0000","last_fullsized":"2024-09-12T02:47:52.404520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223247+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:18.223247+0000","last_peered":"2024-09-12T02:48:18.223247+0000","last_clean":"2024-09-12T02:48:18.223247+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:18.223247+0000","last_undegraded":"2024-09-12T02:48:18.223247+0000","last_fullsized":"2024-09-12T02:48:18.223247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405303+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:47:52.405303+0000","last_peered":"2024-09-12T02:47:52.405303+0000","last_clean":"2024-09-12T02:47:52.405303+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:47:52.405303+0000","last_undegraded":"2024-09-12T02:47:52.405303+0000","last_fullsized":"2024-09-12T02:47:52.405303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403866+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:47:52.403866+0000","last_peered":"2024-09-12T02:47:52.403866+0000","last_clean":"2024-09-12T02:47:52.403866+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:47:52.403866+0000","last_undegraded":"2024-09-12T02:47:52.403866+0000","last_fullsized":"2024-09-12T02:47:52.403866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223376+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:18.223376+0000","last_peered":"2024-09-12T02:48:18.223376+0000","last_clean":"2024-09-12T02:48:18.223376+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:18.223376+0000","last_undegraded":"2024-09-12T02:48:18.223376+0000","last_fullsized":"2024-09-12T02:48:18.223376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141198+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:47:51.141198+0000","last_peered":"2024-09-12T02:47:51.141198+0000","last_clean":"2024-09-12T02:47:51.141198+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:47:51.141198+0000","last_undegraded":"2024-09-12T02:47:51.141198+0000","last_fullsized":"2024-09-12T02:47:51.141198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405159+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:47:52.405159+0000","last_peered":"2024-09-12T02:47:52.405159+0000","last_clean":"2024-09-12T02:47:52.405159+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:47:52.405159+0000","last_undegraded":"2024-09-12T02:47:52.405159+0000","last_fullsized":"2024-09-12T02:47:52.405159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:30.120263+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:48:30.120263+0000","last_peered":"2024-09-12T02:48:30.120263+0000","last_clean":"2024-09-12T02:48:30.120263+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:48:30.120263+0000","last_undegraded":"2024-09-12T02:48:30.120263+0000","last_fullsized":"2024-09-12T02:48:30.120263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281217+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:47:52.281217+0000","last_peered":"2024-09-12T02:47:52.281217+0000","last_clean":"2024-09-12T02:47:52.281217+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:47:52.281217+0000","last_undegraded":"2024-09-12T02:47:52.281217+0000","last_fullsized":"2024-09-12T02:47:52.281217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403758+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:47:52.403758+0000","last_peered":"2024-09-12T02:47:52.403758+0000","last_clean":"2024-09-12T02:47:52.403758+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:47:52.403758+0000","last_undegraded":"2024-09-12T02:47:52.403758+0000","last_fullsized":"2024-09-12T02:47:52.403758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141215+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:47:51.141215+0000","last_peered":"2024-09-12T02:47:51.141215+0000","last_clean":"2024-09-12T02:47:51.141215+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:47:51.141215+0000","last_undegraded":"2024-09-12T02:47:51.141215+0000","last_fullsized":"2024-09-12T02:47:51.141215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226564+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:47:52.226564+0000","last_peered":"2024-09-12T02:47:52.226564+0000","last_clean":"2024-09-12T02:47:52.226564+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:47:52.226564+0000","last_undegraded":"2024-09-12T02:47:52.226564+0000","last_fullsized":"2024-09-12T02:47:52.226564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141988+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:47:51.141988+0000","last_peered":"2024-09-12T02:47:51.141988+0000","last_clean":"2024-09-12T02:47:51.141988+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:47:51.141988+0000","last_undegraded":"2024-09-12T02:47:51.141988+0000","last_fullsized":"2024-09-12T02:47:51.141988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281131+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:47:52.281131+0000","last_peered":"2024-09-12T02:47:52.281131+0000","last_clean":"2024-09-12T02:47:52.281131+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:47:52.281131+0000","last_undegraded":"2024-09-12T02:47:52.281131+0000","last_fullsized":"2024-09-12T02:47:52.281131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234186+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:19.234186+0000","last_peered":"2024-09-12T02:48:19.234186+0000","last_clean":"2024-09-12T02:48:19.234186+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:19.234186+0000","last_undegraded":"2024-09-12T02:48:19.234186+0000","last_fullsized":"2024-09-12T02:48:19.234186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234039+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:19.234039+0000","last_peered":"2024-09-12T02:48:19.234039+0000","last_clean":"2024-09-12T02:48:19.234039+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:19.234039+0000","last_undegraded":"2024-09-12T02:48:19.234039+0000","last_fullsized":"2024-09-12T02:48:19.234039+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226415+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:47:52.226415+0000","last_peered":"2024-09-12T02:47:52.226415+0000","last_clean":"2024-09-12T02:47:52.226415+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:47:52.226415+0000","last_undegraded":"2024-09-12T02:47:52.226415+0000","last_fullsized":"2024-09-12T02:47:52.226415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403756+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:47:52.403756+0000","last_peered":"2024-09-12T02:47:52.403756+0000","last_clean":"2024-09-12T02:47:52.403756+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:47:52.403756+0000","last_undegraded":"2024-09-12T02:47:52.403756+0000","last_fullsized":"2024-09-12T02:47:52.403756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:31.122686+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:48:31.122686+0000","last_peered":"2024-09-12T02:48:31.122686+0000","last_clean":"2024-09-12T02:48:31.122686+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:48:31.122686+0000","last_undegraded":"2024-09-12T02:48:31.122686+0000","last_fullsized":"2024-09-12T02:48:31.122686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404036+0000","last_change":"2024-09-12T02:47:41.114331+0000","last_active":"2024-09-12T02:47:52.404036+0000","last_peered":"2024-09-12T02:47:52.404036+0000","last_clean":"2024-09-12T02:47:52.404036+0000","last_became_active":"2024-09-12T02:47:41.113977+0000","last_became_peered":"2024-09-12T02:47:41.113977+0000","last_unstale":"2024-09-12T02:47:52.404036+0000","last_undegraded":"2024-09-12T02:47:52.404036+0000","last_fullsized":"2024-09-12T02:47:52.404036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:41.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403828+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:47:52.403828+0000","last_peered":"2024-09-12T02:47:52.403828+0000","last_clean":"2024-09-12T02:47:52.403828+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:47:52.403828+0000","last_undegraded":"2024-09-12T02:47:52.403828+0000","last_fullsized":"2024-09-12T02:47:52.403828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141425+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:47:51.141425+0000","last_peered":"2024-09-12T02:47:51.141425+0000","last_clean":"2024-09-12T02:47:51.141425+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:47:51.141425+0000","last_undegraded":"2024-09-12T02:47:51.141425+0000","last_fullsized":"2024-09-12T02:47:51.141425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403665+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:47:52.403665+0000","last_peered":"2024-09-12T02:47:52.403665+0000","last_clean":"2024-09-12T02:47:52.403665+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:47:52.403665+0000","last_undegraded":"2024-09-12T02:47:52.403665+0000","last_fullsized":"2024-09-12T02:47:52.403665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403447+0000","last_change":"2024-09-12T02:47:41.119469+0000","last_active":"2024-09-12T02:47:52.403447+0000","last_peered":"2024-09-12T02:47:52.403447+0000","last_clean":"2024-09-12T02:47:52.403447+0000","last_became_active":"2024-09-12T02:47:41.119120+0000","last_became_peered":"2024-09-12T02:47:41.119120+0000","last_unstale":"2024-09-12T02:47:52.403447+0000","last_undegraded":"2024-09-12T02:47:52.403447+0000","last_fullsized":"2024-09-12T02:47:52.403447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:27.297002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403525+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:47:52.403525+0000","last_peered":"2024-09-12T02:47:52.403525+0000","last_clean":"2024-09-12T02:47:52.403525+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:47:52.403525+0000","last_undegraded":"2024-09-12T02:47:52.403525+0000","last_fullsized":"2024-09-12T02:47:52.403525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499329+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:47:52.499329+0000","last_peered":"2024-09-12T02:47:52.499329+0000","last_clean":"2024-09-12T02:47:52.499329+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:47:52.499329+0000","last_undegraded":"2024-09-12T02:47:52.499329+0000","last_fullsized":"2024-09-12T02:47:52.499329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141883+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:47:51.141883+0000","last_peered":"2024-09-12T02:47:51.141883+0000","last_clean":"2024-09-12T02:47:51.141883+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:47:51.141883+0000","last_undegraded":"2024-09-12T02:47:51.141883+0000","last_fullsized":"2024-09-12T02:47:51.141883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403901+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:47:52.403901+0000","last_peered":"2024-09-12T02:47:52.403901+0000","last_clean":"2024-09-12T02:47:52.403901+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:47:52.403901+0000","last_undegraded":"2024-09-12T02:47:52.403901+0000","last_fullsized":"2024-09-12T02:47:52.403901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281451+0000","last_change":"2024-09-12T02:47:41.118766+0000","last_active":"2024-09-12T02:47:52.281451+0000","last_peered":"2024-09-12T02:47:52.281451+0000","last_clean":"2024-09-12T02:47:52.281451+0000","last_became_active":"2024-09-12T02:47:41.118459+0000","last_became_peered":"2024-09-12T02:47:41.118459+0000","last_unstale":"2024-09-12T02:47:52.281451+0000","last_undegraded":"2024-09-12T02:47:52.281451+0000","last_fullsized":"2024-09-12T02:47:52.281451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:12.983552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.226232+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:47:52.226232+0000","last_peered":"2024-09-12T02:47:52.226232+0000","last_clean":"2024-09-12T02:47:52.226232+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:47:52.226232+0000","last_undegraded":"2024-09-12T02:47:52.226232+0000","last_fullsized":"2024-09-12T02:47:52.226232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281304+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:47:52.281304+0000","last_peered":"2024-09-12T02:47:52.281304+0000","last_clean":"2024-09-12T02:47:52.281304+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:47:52.281304+0000","last_undegraded":"2024-09-12T02:47:52.281304+0000","last_fullsized":"2024-09-12T02:47:52.281304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404536+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:47:52.404536+0000","last_peered":"2024-09-12T02:47:52.404536+0000","last_clean":"2024-09-12T02:47:52.404536+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:47:52.404536+0000","last_undegraded":"2024-09-12T02:47:52.404536+0000","last_fullsized":"2024-09-12T02:47:52.404536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499229+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:47:52.499229+0000","last_peered":"2024-09-12T02:47:52.499229+0000","last_clean":"2024-09-12T02:47:52.499229+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:47:52.499229+0000","last_undegraded":"2024-09-12T02:47:52.499229+0000","last_fullsized":"2024-09-12T02:47:52.499229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141687+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:47:51.141687+0000","last_peered":"2024-09-12T02:47:51.141687+0000","last_clean":"2024-09-12T02:47:51.141687+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:47:51.141687+0000","last_undegraded":"2024-09-12T02:47:51.141687+0000","last_fullsized":"2024-09-12T02:47:51.141687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403866+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:47:52.403866+0000","last_peered":"2024-09-12T02:47:52.403866+0000","last_clean":"2024-09-12T02:47:52.403866+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:47:52.403866+0000","last_undegraded":"2024-09-12T02:47:52.403866+0000","last_fullsized":"2024-09-12T02:47:52.403866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141850+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:47:51.141850+0000","last_peered":"2024-09-12T02:47:51.141850+0000","last_clean":"2024-09-12T02:47:51.141850+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:47:51.141850+0000","last_undegraded":"2024-09-12T02:47:51.141850+0000","last_fullsized":"2024-09-12T02:47:51.141850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.142126+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:47:51.142126+0000","last_peered":"2024-09-12T02:47:51.142126+0000","last_clean":"2024-09-12T02:47:51.142126+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:47:51.142126+0000","last_undegraded":"2024-09-12T02:47:51.142126+0000","last_fullsized":"2024-09-12T02:47:51.142126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403793+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:47:52.403793+0000","last_peered":"2024-09-12T02:47:52.403793+0000","last_clean":"2024-09-12T02:47:52.403793+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:47:52.403793+0000","last_undegraded":"2024-09-12T02:47:52.403793+0000","last_fullsized":"2024-09-12T02:47:52.403793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403453+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:47:52.403453+0000","last_peered":"2024-09-12T02:47:52.403453+0000","last_clean":"2024-09-12T02:47:52.403453+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:47:52.403453+0000","last_undegraded":"2024-09-12T02:47:52.403453+0000","last_fullsized":"2024-09-12T02:47:52.403453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403460+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:47:52.403460+0000","last_peered":"2024-09-12T02:47:52.403460+0000","last_clean":"2024-09-12T02:47:52.403460+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:47:52.403460+0000","last_undegraded":"2024-09-12T02:47:52.403460+0000","last_fullsized":"2024-09-12T02:47:52.403460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499055+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:47:52.499055+0000","last_peered":"2024-09-12T02:47:52.499055+0000","last_clean":"2024-09-12T02:47:52.499055+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:47:52.499055+0000","last_undegraded":"2024-09-12T02:47:52.499055+0000","last_fullsized":"2024-09-12T02:47:52.499055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405267+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:47:52.405267+0000","last_peered":"2024-09-12T02:47:52.405267+0000","last_clean":"2024-09-12T02:47:52.405267+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:47:52.405267+0000","last_undegraded":"2024-09-12T02:47:52.405267+0000","last_fullsized":"2024-09-12T02:47:52.405267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405974+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:47:52.405974+0000","last_peered":"2024-09-12T02:47:52.405974+0000","last_clean":"2024-09-12T02:47:52.405974+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:47:52.405974+0000","last_undegraded":"2024-09-12T02:47:52.405974+0000","last_fullsized":"2024-09-12T02:47:52.405974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141982+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:47:51.141982+0000","last_peered":"2024-09-12T02:47:51.141982+0000","last_clean":"2024-09-12T02:47:51.141982+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:47:51.141982+0000","last_undegraded":"2024-09-12T02:47:51.141982+0000","last_fullsized":"2024-09-12T02:47:51.141982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405199+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:47:52.405199+0000","last_peered":"2024-09-12T02:47:52.405199+0000","last_clean":"2024-09-12T02:47:52.405199+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:47:52.405199+0000","last_undegraded":"2024-09-12T02:47:52.405199+0000","last_fullsized":"2024-09-12T02:47:52.405199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281004+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:47:52.281004+0000","last_peered":"2024-09-12T02:47:52.281004+0000","last_clean":"2024-09-12T02:47:52.281004+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:47:52.281004+0000","last_undegraded":"2024-09-12T02:47:52.281004+0000","last_fullsized":"2024-09-12T02:47:52.281004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404645+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:47:52.404645+0000","last_peered":"2024-09-12T02:47:52.404645+0000","last_clean":"2024-09-12T02:47:52.404645+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:47:52.404645+0000","last_undegraded":"2024-09-12T02:47:52.404645+0000","last_fullsized":"2024-09-12T02:47:52.404645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141621+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:47:51.141621+0000","last_peered":"2024-09-12T02:47:51.141621+0000","last_clean":"2024-09-12T02:47:51.141621+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:47:51.141621+0000","last_undegraded":"2024-09-12T02:47:51.141621+0000","last_fullsized":"2024-09-12T02:47:51.141621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403861+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:47:52.403861+0000","last_peered":"2024-09-12T02:47:52.403861+0000","last_clean":"2024-09-12T02:47:52.403861+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:47:52.403861+0000","last_undegraded":"2024-09-12T02:47:52.403861+0000","last_fullsized":"2024-09-12T02:47:52.403861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405153+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:47:52.405153+0000","last_peered":"2024-09-12T02:47:52.405153+0000","last_clean":"2024-09-12T02:47:52.405153+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:47:52.405153+0000","last_undegraded":"2024-09-12T02:47:52.405153+0000","last_fullsized":"2024-09-12T02:47:52.405153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403725+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:47:52.403725+0000","last_peered":"2024-09-12T02:47:52.403725+0000","last_clean":"2024-09-12T02:47:52.403725+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:47:52.403725+0000","last_undegraded":"2024-09-12T02:47:52.403725+0000","last_fullsized":"2024-09-12T02:47:52.403725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403733+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:47:52.403733+0000","last_peered":"2024-09-12T02:47:52.403733+0000","last_clean":"2024-09-12T02:47:52.403733+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:47:52.403733+0000","last_undegraded":"2024-09-12T02:47:52.403733+0000","last_fullsized":"2024-09-12T02:47:52.403733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.142059+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:47:51.142059+0000","last_peered":"2024-09-12T02:47:51.142059+0000","last_clean":"2024-09-12T02:47:51.142059+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:47:51.142059+0000","last_undegraded":"2024-09-12T02:47:51.142059+0000","last_fullsized":"2024-09-12T02:47:51.142059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":96,"num_read_kb":98,"num_write":153,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":153095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":482479,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743452,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":1113092,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906978,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":486811,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":478539,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1078546,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528368,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":477891,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1074134,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":482541,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":478085,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":508755,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":532197,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":479279,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:48:36.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:48:37.213 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-12T02:48:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-12T02:48:37.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:37 smithi145 ceph-mon[34674]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:48:37.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:37.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T02:48:37.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-12T02:48:37.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:37 smithi046 ceph-mon[30951]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:48:37.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-12T02:48:37.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-12T02:48:37.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:37 smithi196 ceph-mon[34587]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:48:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- 172.21.15.46:0/2113408283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98101de0 msgr2=0x7fce981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 --2- 172.21.15.46:0/2113408283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98101de0 0x7fce981021c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce94009900 tx=0x7fce9402a410 comp rx=0 tx=0).stop 2024-09-12T02:48:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- 172.21.15.46:0/2113408283 shutdown_connections 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 --2- 172.21.15.46:0/2113408283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98106f10 0x7fce9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 --2- 172.21.15.46:0/2113408283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98102790 0x7fce981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 --2- 172.21.15.46:0/2113408283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98101de0 0x7fce981021c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- 172.21.15.46:0/2113408283 >> 172.21.15.46:0/2113408283 conn(0x7fce980fd8c0 msgr2=0x7fce980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- 172.21.15.46:0/2113408283 shutdown_connections 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- 172.21.15.46:0/2113408283 wait complete. 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 Processor -- start 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.723+0000 7fcea068b640 1 -- start start 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 0x7fce9819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 0x7fce9819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce98111e20 con 0x7fce98102790 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce98111ca0 con 0x7fce98106f10 2024-09-12T02:48:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fcea068b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce98111fa0 con 0x7fce98101de0 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52428/0 (socket says 172.21.15.46:52428) 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 -- 172.21.15.46:0/155123548 learned_addr learned my addr 172.21.15.46:0/155123548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9e427640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 0x7fce9819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 -- 172.21.15.46:0/155123548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 msgr2=0x7fce9819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 0x7fce9819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 -- 172.21.15.46:0/155123548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 msgr2=0x7fce9819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9ec28640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 0x7fce9819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 0x7fce9819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 -- 172.21.15.46:0/155123548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9819f5b0 con 0x7fce98102790 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9e427640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 0x7fce9819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:37.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9ec28640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 0x7fce9819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.724+0000 7fce9dc26640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce8800d950 tx=0x7fce8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.725+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88017070 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.725+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9819f8a0 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.725+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce88009c30 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.725+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8800f620 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.725+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce981a7180 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.726+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce981021c0 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.727+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce880109a0 con 0x7fce98102790 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.727+0000 7fce8f7fe640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 0x7fce840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.728+0000 7fce9e427640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 0x7fce840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.728+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fce880cf1f0 con 0x7fce98102790 2024-09-12T02:48:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.728+0000 7fce9e427640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 0x7fce840a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce94031440 tx=0x7fce94005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.730+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce88009de0 con 0x7fce98102790 2024-09-12T02:48:37.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.904+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce98075430 con 0x7fce840a2c10 2024-09-12T02:48:37.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.914+0000 7fce8f7fe640 1 -- 172.21.15.46:0/155123548 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340524 (secure 0 0 0) 0x7fce98075430 con 0x7fce840a2c10 2024-09-12T02:48:37.918 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:48:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 msgr2=0x7fce840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 0x7fce840a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce94031440 tx=0x7fce94005a20 comp rx=0 tx=0).stop 2024-09-12T02:48:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 msgr2=0x7fce9819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fce8800d950 tx=0x7fce8800de10 comp rx=0 tx=0).stop 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 shutdown_connections 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce98106f10 0x7fce9819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce98102790 0x7fce9819ab40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fce840a2c10 0x7fce840a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 --2- 172.21.15.46:0/155123548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce98101de0 0x7fce9819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 >> 172.21.15.46:0/155123548 conn(0x7fce980fd8c0 msgr2=0x7fce980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 shutdown_connections 2024-09-12T02:48:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:37.918+0000 7fcea068b640 1 -- 172.21.15.46:0/155123548 wait complete. 2024-09-12T02:48:38.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:38 smithi046 ceph-mon[30951]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:48:38.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:48:38.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:38 smithi046 ceph-mon[30951]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:38 smithi145 ceph-mon[34674]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:48:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:48:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:38 smithi145 ceph-mon[34674]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:38.674 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:38 smithi196 ceph-mon[34587]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:48:38.674 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:48:38.674 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:38 smithi196 ceph-mon[34587]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:38.701 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-12T02:48:37.334926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649001,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":263,"num_read_kb":243,"num_write":347,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369308,"kb_used_data":11732,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572516,"statfs":{"total":1151940427776,"available":1151562256384,"internally_reserved":0,"allocated":12013568,"data_stored":7731845,"data_compressed":325940,"data_compressed_allocated":438272,"data_compressed_original":2375680,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29933,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":60,"num_write":23,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.064531"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336042+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:36.336042+0000","last_peered":"2024-09-12T02:48:36.336042+0000","last_clean":"2024-09-12T02:48:36.336042+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:36.336042+0000","last_undegraded":"2024-09-12T02:48:36.336042+0000","last_fullsized":"2024-09-12T02:48:36.336042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281137+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:47:52.281137+0000","last_peered":"2024-09-12T02:47:52.281137+0000","last_clean":"2024-09-12T02:47:52.281137+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:47:52.281137+0000","last_undegraded":"2024-09-12T02:47:52.281137+0000","last_fullsized":"2024-09-12T02:47:52.281137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403462+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:47:52.403462+0000","last_peered":"2024-09-12T02:47:52.403462+0000","last_clean":"2024-09-12T02:47:52.403462+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:47:52.403462+0000","last_undegraded":"2024-09-12T02:47:52.403462+0000","last_fullsized":"2024-09-12T02:47:52.403462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281434+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:47:52.281434+0000","last_peered":"2024-09-12T02:47:52.281434+0000","last_clean":"2024-09-12T02:47:52.281434+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:47:52.281434+0000","last_undegraded":"2024-09-12T02:47:52.281434+0000","last_fullsized":"2024-09-12T02:47:52.281434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403846+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:47:52.403846+0000","last_peered":"2024-09-12T02:47:52.403846+0000","last_clean":"2024-09-12T02:47:52.403846+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:47:52.403846+0000","last_undegraded":"2024-09-12T02:47:52.403846+0000","last_fullsized":"2024-09-12T02:47:52.403846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.337863+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:36.337863+0000","last_peered":"2024-09-12T02:48:36.337863+0000","last_clean":"2024-09-12T02:48:36.337863+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:36.337863+0000","last_undegraded":"2024-09-12T02:48:36.337863+0000","last_fullsized":"2024-09-12T02:48:36.337863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234397+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:19.234397+0000","last_peered":"2024-09-12T02:48:19.234397+0000","last_clean":"2024-09-12T02:48:19.234397+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:19.234397+0000","last_undegraded":"2024-09-12T02:48:19.234397+0000","last_fullsized":"2024-09-12T02:48:19.234397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404446+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:47:52.404446+0000","last_peered":"2024-09-12T02:47:52.404446+0000","last_clean":"2024-09-12T02:47:52.404446+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:47:52.404446+0000","last_undegraded":"2024-09-12T02:47:52.404446+0000","last_fullsized":"2024-09-12T02:47:52.404446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338890+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:36.338890+0000","last_peered":"2024-09-12T02:48:36.338890+0000","last_clean":"2024-09-12T02:48:36.338890+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:36.338890+0000","last_undegraded":"2024-09-12T02:48:36.338890+0000","last_fullsized":"2024-09-12T02:48:36.338890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404295+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:47:52.404295+0000","last_peered":"2024-09-12T02:47:52.404295+0000","last_clean":"2024-09-12T02:47:52.404295+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:47:52.404295+0000","last_undegraded":"2024-09-12T02:47:52.404295+0000","last_fullsized":"2024-09-12T02:47:52.404295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340445+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:36.340445+0000","last_peered":"2024-09-12T02:48:36.340445+0000","last_clean":"2024-09-12T02:48:36.340445+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:36.340445+0000","last_undegraded":"2024-09-12T02:48:36.340445+0000","last_fullsized":"2024-09-12T02:48:36.340445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338032+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:36.338032+0000","last_peered":"2024-09-12T02:48:36.338032+0000","last_clean":"2024-09-12T02:48:36.338032+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:36.338032+0000","last_undegraded":"2024-09-12T02:48:36.338032+0000","last_fullsized":"2024-09-12T02:48:36.338032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405358+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:47:52.405358+0000","last_peered":"2024-09-12T02:47:52.405358+0000","last_clean":"2024-09-12T02:47:52.405358+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:47:52.405358+0000","last_undegraded":"2024-09-12T02:47:52.405358+0000","last_fullsized":"2024-09-12T02:47:52.405358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.222528+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:18.222528+0000","last_peered":"2024-09-12T02:48:18.222528+0000","last_clean":"2024-09-12T02:48:18.222528+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:18.222528+0000","last_undegraded":"2024-09-12T02:48:18.222528+0000","last_fullsized":"2024-09-12T02:48:18.222528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338314+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:36.338314+0000","last_peered":"2024-09-12T02:48:36.338314+0000","last_clean":"2024-09-12T02:48:36.338314+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:36.338314+0000","last_undegraded":"2024-09-12T02:48:36.338314+0000","last_fullsized":"2024-09-12T02:48:36.338314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280980+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:47:52.280980+0000","last_peered":"2024-09-12T02:47:52.280980+0000","last_clean":"2024-09-12T02:47:52.280980+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:47:52.280980+0000","last_undegraded":"2024-09-12T02:47:52.280980+0000","last_fullsized":"2024-09-12T02:47:52.280980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403643+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:47:52.403643+0000","last_peered":"2024-09-12T02:47:52.403643+0000","last_clean":"2024-09-12T02:47:52.403643+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:47:52.403643+0000","last_undegraded":"2024-09-12T02:47:52.403643+0000","last_fullsized":"2024-09-12T02:47:52.403643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338333+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:36.338333+0000","last_peered":"2024-09-12T02:48:36.338333+0000","last_clean":"2024-09-12T02:48:36.338333+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:36.338333+0000","last_undegraded":"2024-09-12T02:48:36.338333+0000","last_fullsized":"2024-09-12T02:48:36.338333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141492+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:47:51.141492+0000","last_peered":"2024-09-12T02:47:51.141492+0000","last_clean":"2024-09-12T02:47:51.141492+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:47:51.141492+0000","last_undegraded":"2024-09-12T02:47:51.141492+0000","last_fullsized":"2024-09-12T02:47:51.141492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223309+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:18.223309+0000","last_peered":"2024-09-12T02:48:18.223309+0000","last_clean":"2024-09-12T02:48:18.223309+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:18.223309+0000","last_undegraded":"2024-09-12T02:48:18.223309+0000","last_fullsized":"2024-09-12T02:48:18.223309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335865+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:36.335865+0000","last_peered":"2024-09-12T02:48:36.335865+0000","last_clean":"2024-09-12T02:48:36.335865+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:36.335865+0000","last_undegraded":"2024-09-12T02:48:36.335865+0000","last_fullsized":"2024-09-12T02:48:36.335865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403682+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:47:52.403682+0000","last_peered":"2024-09-12T02:47:52.403682+0000","last_clean":"2024-09-12T02:47:52.403682+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:47:52.403682+0000","last_undegraded":"2024-09-12T02:47:52.403682+0000","last_fullsized":"2024-09-12T02:47:52.403682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223316+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:18.223316+0000","last_peered":"2024-09-12T02:48:18.223316+0000","last_clean":"2024-09-12T02:48:18.223316+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:18.223316+0000","last_undegraded":"2024-09-12T02:48:18.223316+0000","last_fullsized":"2024-09-12T02:48:18.223316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-09-12T02:47:52.404393+0000","last_change":"2024-09-12T02:48:36.338444+0000","last_active":"2024-09-12T02:48:36.338444+0000","last_peered":"2024-09-12T02:47:52.404393+0000","last_clean":"2024-09-12T02:47:52.404393+0000","last_became_active":"2024-09-12T02:47:48.158833+0000","last_became_peered":"2024-09-12T02:47:48.158833+0000","last_unstale":"2024-09-12T02:47:52.404393+0000","last_undegraded":"2024-09-12T02:47:52.404393+0000","last_fullsized":"2024-09-12T02:47:52.404393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403831+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:47:52.403831+0000","last_peered":"2024-09-12T02:47:52.403831+0000","last_clean":"2024-09-12T02:47:52.403831+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:47:52.403831+0000","last_undegraded":"2024-09-12T02:47:52.403831+0000","last_fullsized":"2024-09-12T02:47:52.403831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340605+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:36.340605+0000","last_peered":"2024-09-12T02:48:36.340605+0000","last_clean":"2024-09-12T02:48:36.340605+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:36.340605+0000","last_undegraded":"2024-09-12T02:48:36.340605+0000","last_fullsized":"2024-09-12T02:48:36.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403534+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:47:52.403534+0000","last_peered":"2024-09-12T02:47:52.403534+0000","last_clean":"2024-09-12T02:47:52.403534+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:47:52.403534+0000","last_undegraded":"2024-09-12T02:47:52.403534+0000","last_fullsized":"2024-09-12T02:47:52.403534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281499+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:47:52.281499+0000","last_peered":"2024-09-12T02:47:52.281499+0000","last_clean":"2024-09-12T02:47:52.281499+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:47:52.281499+0000","last_undegraded":"2024-09-12T02:47:52.281499+0000","last_fullsized":"2024-09-12T02:47:52.281499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336358+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:36.336358+0000","last_peered":"2024-09-12T02:48:36.336358+0000","last_clean":"2024-09-12T02:48:36.336358+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:36.336358+0000","last_undegraded":"2024-09-12T02:48:36.336358+0000","last_fullsized":"2024-09-12T02:48:36.336358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280833+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:47:52.280833+0000","last_peered":"2024-09-12T02:47:52.280833+0000","last_clean":"2024-09-12T02:47:52.280833+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:47:52.280833+0000","last_undegraded":"2024-09-12T02:47:52.280833+0000","last_fullsized":"2024-09-12T02:47:52.280833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499352+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:47:52.499352+0000","last_peered":"2024-09-12T02:47:52.499352+0000","last_clean":"2024-09-12T02:47:52.499352+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:47:52.499352+0000","last_undegraded":"2024-09-12T02:47:52.499352+0000","last_fullsized":"2024-09-12T02:47:52.499352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.220889+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:18.220889+0000","last_peered":"2024-09-12T02:48:18.220889+0000","last_clean":"2024-09-12T02:48:18.220889+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:18.220889+0000","last_undegraded":"2024-09-12T02:48:18.220889+0000","last_fullsized":"2024-09-12T02:48:18.220889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:47:51.141552+0000","last_peered":"2024-09-12T02:47:51.141552+0000","last_clean":"2024-09-12T02:47:51.141552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:47:51.141552+0000","last_undegraded":"2024-09-12T02:47:51.141552+0000","last_fullsized":"2024-09-12T02:47:51.141552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141747+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:47:51.141747+0000","last_peered":"2024-09-12T02:47:51.141747+0000","last_clean":"2024-09-12T02:47:51.141747+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:47:51.141747+0000","last_undegraded":"2024-09-12T02:47:51.141747+0000","last_fullsized":"2024-09-12T02:47:51.141747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335512+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:36.335512+0000","last_peered":"2024-09-12T02:48:36.335512+0000","last_clean":"2024-09-12T02:48:36.335512+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:36.335512+0000","last_undegraded":"2024-09-12T02:48:36.335512+0000","last_fullsized":"2024-09-12T02:48:36.335512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403573+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:47:52.403573+0000","last_peered":"2024-09-12T02:47:52.403573+0000","last_clean":"2024-09-12T02:47:52.403573+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:47:52.403573+0000","last_undegraded":"2024-09-12T02:47:52.403573+0000","last_fullsized":"2024-09-12T02:47:52.403573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141682+0000","last_change":"2024-09-12T02:47:41.115955+0000","last_active":"2024-09-12T02:47:51.141682+0000","last_peered":"2024-09-12T02:47:51.141682+0000","last_clean":"2024-09-12T02:47:51.141682+0000","last_became_active":"2024-09-12T02:47:41.115604+0000","last_became_peered":"2024-09-12T02:47:41.115604+0000","last_unstale":"2024-09-12T02:47:51.141682+0000","last_undegraded":"2024-09-12T02:47:51.141682+0000","last_fullsized":"2024-09-12T02:47:51.141682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:46.153312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405554+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:47:52.405554+0000","last_peered":"2024-09-12T02:47:52.405554+0000","last_clean":"2024-09-12T02:47:52.405554+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:47:52.405554+0000","last_undegraded":"2024-09-12T02:47:52.405554+0000","last_fullsized":"2024-09-12T02:47:52.405554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499174+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:47:52.499174+0000","last_peered":"2024-09-12T02:47:52.499174+0000","last_clean":"2024-09-12T02:47:52.499174+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:47:52.499174+0000","last_undegraded":"2024-09-12T02:47:52.499174+0000","last_fullsized":"2024-09-12T02:47:52.499174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338309+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:36.338309+0000","last_peered":"2024-09-12T02:48:36.338309+0000","last_clean":"2024-09-12T02:48:36.338309+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:36.338309+0000","last_undegraded":"2024-09-12T02:48:36.338309+0000","last_fullsized":"2024-09-12T02:48:36.338309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:33.124962+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:48:33.124962+0000","last_peered":"2024-09-12T02:48:33.124962+0000","last_clean":"2024-09-12T02:48:33.124962+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:48:33.124962+0000","last_undegraded":"2024-09-12T02:48:33.124962+0000","last_fullsized":"2024-09-12T02:48:33.124962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499186+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:47:52.499186+0000","last_peered":"2024-09-12T02:47:52.499186+0000","last_clean":"2024-09-12T02:47:52.499186+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:47:52.499186+0000","last_undegraded":"2024-09-12T02:47:52.499186+0000","last_fullsized":"2024-09-12T02:47:52.499186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.498800+0000","last_change":"2024-09-12T02:47:41.114262+0000","last_active":"2024-09-12T02:47:52.498800+0000","last_peered":"2024-09-12T02:47:52.498800+0000","last_clean":"2024-09-12T02:47:52.498800+0000","last_became_active":"2024-09-12T02:47:41.113951+0000","last_became_peered":"2024-09-12T02:47:41.113951+0000","last_unstale":"2024-09-12T02:47:52.498800+0000","last_undegraded":"2024-09-12T02:47:52.498800+0000","last_fullsized":"2024-09-12T02:47:52.498800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:38.948691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.233809+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:19.233809+0000","last_peered":"2024-09-12T02:48:19.233809+0000","last_clean":"2024-09-12T02:48:19.233809+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:19.233809+0000","last_undegraded":"2024-09-12T02:48:19.233809+0000","last_fullsized":"2024-09-12T02:48:19.233809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-09-12T02:47:52.226786+0000","last_change":"2024-09-12T02:48:36.338797+0000","last_active":"2024-09-12T02:48:36.338797+0000","last_peered":"2024-09-12T02:47:52.226786+0000","last_clean":"2024-09-12T02:47:52.226786+0000","last_became_active":"2024-09-12T02:47:48.157488+0000","last_became_peered":"2024-09-12T02:47:48.157488+0000","last_unstale":"2024-09-12T02:47:52.226786+0000","last_undegraded":"2024-09-12T02:47:52.226786+0000","last_fullsized":"2024-09-12T02:47:52.226786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,8,9,10],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281134+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:47:52.281134+0000","last_peered":"2024-09-12T02:47:52.281134+0000","last_clean":"2024-09-12T02:47:52.281134+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:47:52.281134+0000","last_undegraded":"2024-09-12T02:47:52.281134+0000","last_fullsized":"2024-09-12T02:47:52.281134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339118+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:36.339118+0000","last_peered":"2024-09-12T02:48:36.339118+0000","last_clean":"2024-09-12T02:48:36.339118+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:36.339118+0000","last_undegraded":"2024-09-12T02:48:36.339118+0000","last_fullsized":"2024-09-12T02:48:36.339118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405158+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:47:52.405158+0000","last_peered":"2024-09-12T02:47:52.405158+0000","last_clean":"2024-09-12T02:47:52.405158+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:47:52.405158+0000","last_undegraded":"2024-09-12T02:47:52.405158+0000","last_fullsized":"2024-09-12T02:47:52.405158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405866+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:47:52.405866+0000","last_peered":"2024-09-12T02:47:52.405866+0000","last_clean":"2024-09-12T02:47:52.405866+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:47:52.405866+0000","last_undegraded":"2024-09-12T02:47:52.405866+0000","last_fullsized":"2024-09-12T02:47:52.405866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.406006+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:47:52.406006+0000","last_peered":"2024-09-12T02:47:52.406006+0000","last_clean":"2024-09-12T02:47:52.406006+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:47:52.406006+0000","last_undegraded":"2024-09-12T02:47:52.406006+0000","last_fullsized":"2024-09-12T02:47:52.406006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.235709+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:19.235709+0000","last_peered":"2024-09-12T02:48:19.235709+0000","last_clean":"2024-09-12T02:48:19.235709+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:19.235709+0000","last_undegraded":"2024-09-12T02:48:19.235709+0000","last_fullsized":"2024-09-12T02:48:19.235709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404343+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:47:52.404343+0000","last_peered":"2024-09-12T02:47:52.404343+0000","last_clean":"2024-09-12T02:47:52.404343+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:47:52.404343+0000","last_undegraded":"2024-09-12T02:47:52.404343+0000","last_fullsized":"2024-09-12T02:47:52.404343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405711+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:47:52.405711+0000","last_peered":"2024-09-12T02:47:52.405711+0000","last_clean":"2024-09-12T02:47:52.405711+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:47:52.405711+0000","last_undegraded":"2024-09-12T02:47:52.405711+0000","last_fullsized":"2024-09-12T02:47:52.405711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340757+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:36.340757+0000","last_peered":"2024-09-12T02:48:36.340757+0000","last_clean":"2024-09-12T02:48:36.340757+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:36.340757+0000","last_undegraded":"2024-09-12T02:48:36.340757+0000","last_fullsized":"2024-09-12T02:48:36.340757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403541+0000","last_change":"2024-09-12T02:47:48.153637+0000","last_active":"2024-09-12T02:47:52.403541+0000","last_peered":"2024-09-12T02:47:52.403541+0000","last_clean":"2024-09-12T02:47:52.403541+0000","last_became_active":"2024-09-12T02:47:48.153180+0000","last_became_peered":"2024-09-12T02:47:48.153180+0000","last_unstale":"2024-09-12T02:47:52.403541+0000","last_undegraded":"2024-09-12T02:47:52.403541+0000","last_fullsized":"2024-09-12T02:47:52.403541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:59.876118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405875+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:47:52.405875+0000","last_peered":"2024-09-12T02:47:52.405875+0000","last_clean":"2024-09-12T02:47:52.405875+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:47:52.405875+0000","last_undegraded":"2024-09-12T02:47:52.405875+0000","last_fullsized":"2024-09-12T02:47:52.405875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499252+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:47:52.499252+0000","last_peered":"2024-09-12T02:47:52.499252+0000","last_clean":"2024-09-12T02:47:52.499252+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:47:52.499252+0000","last_undegraded":"2024-09-12T02:47:52.499252+0000","last_fullsized":"2024-09-12T02:47:52.499252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338132+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:36.338132+0000","last_peered":"2024-09-12T02:48:36.338132+0000","last_clean":"2024-09-12T02:48:36.338132+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:36.338132+0000","last_undegraded":"2024-09-12T02:48:36.338132+0000","last_fullsized":"2024-09-12T02:48:36.338132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141930+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:47:51.141930+0000","last_peered":"2024-09-12T02:47:51.141930+0000","last_clean":"2024-09-12T02:47:51.141930+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:47:51.141930+0000","last_undegraded":"2024-09-12T02:47:51.141930+0000","last_fullsized":"2024-09-12T02:47:51.141930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:20.104615+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:20.104615+0000","last_peered":"2024-09-12T02:48:20.104615+0000","last_clean":"2024-09-12T02:48:20.104615+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:20.104615+0000","last_undegraded":"2024-09-12T02:48:20.104615+0000","last_fullsized":"2024-09-12T02:48:20.104615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338469+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:36.338469+0000","last_peered":"2024-09-12T02:48:36.338469+0000","last_clean":"2024-09-12T02:48:36.338469+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:36.338469+0000","last_undegraded":"2024-09-12T02:48:36.338469+0000","last_fullsized":"2024-09-12T02:48:36.338469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338359+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:36.338359+0000","last_peered":"2024-09-12T02:48:36.338359+0000","last_clean":"2024-09-12T02:48:36.338359+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:36.338359+0000","last_undegraded":"2024-09-12T02:48:36.338359+0000","last_fullsized":"2024-09-12T02:48:36.338359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.236279+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:19.236279+0000","last_peered":"2024-09-12T02:48:19.236279+0000","last_clean":"2024-09-12T02:48:19.236279+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:19.236279+0000","last_undegraded":"2024-09-12T02:48:19.236279+0000","last_fullsized":"2024-09-12T02:48:19.236279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499277+0000","last_change":"2024-09-12T02:47:48.149728+0000","last_active":"2024-09-12T02:47:52.499277+0000","last_peered":"2024-09-12T02:47:52.499277+0000","last_clean":"2024-09-12T02:47:52.499277+0000","last_became_active":"2024-09-12T02:47:48.149314+0000","last_became_peered":"2024-09-12T02:47:48.149314+0000","last_unstale":"2024-09-12T02:47:52.499277+0000","last_undegraded":"2024-09-12T02:47:52.499277+0000","last_fullsized":"2024-09-12T02:47:52.499277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:34.293981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404831+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:47:52.404831+0000","last_peered":"2024-09-12T02:47:52.404831+0000","last_clean":"2024-09-12T02:47:52.404831+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:47:52.404831+0000","last_undegraded":"2024-09-12T02:47:52.404831+0000","last_fullsized":"2024-09-12T02:47:52.404831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338500+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:48:36.338500+0000","last_peered":"2024-09-12T02:48:36.338500+0000","last_clean":"2024-09-12T02:48:36.338500+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:48:36.338500+0000","last_undegraded":"2024-09-12T02:48:36.338500+0000","last_fullsized":"2024-09-12T02:48:36.338500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335854+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:36.335854+0000","last_peered":"2024-09-12T02:48:36.335854+0000","last_clean":"2024-09-12T02:48:36.335854+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:36.335854+0000","last_undegraded":"2024-09-12T02:48:36.335854+0000","last_fullsized":"2024-09-12T02:48:36.335854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405675+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:47:52.405675+0000","last_peered":"2024-09-12T02:47:52.405675+0000","last_clean":"2024-09-12T02:47:52.405675+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:47:52.405675+0000","last_undegraded":"2024-09-12T02:47:52.405675+0000","last_fullsized":"2024-09-12T02:47:52.405675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339970+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:36.339970+0000","last_peered":"2024-09-12T02:48:36.339970+0000","last_clean":"2024-09-12T02:48:36.339970+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:36.339970+0000","last_undegraded":"2024-09-12T02:48:36.339970+0000","last_fullsized":"2024-09-12T02:48:36.339970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141558+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:47:51.141558+0000","last_peered":"2024-09-12T02:47:51.141558+0000","last_clean":"2024-09-12T02:47:51.141558+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:47:51.141558+0000","last_undegraded":"2024-09-12T02:47:51.141558+0000","last_fullsized":"2024-09-12T02:47:51.141558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.498955+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:47:52.498955+0000","last_peered":"2024-09-12T02:47:52.498955+0000","last_clean":"2024-09-12T02:47:52.498955+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:47:52.498955+0000","last_undegraded":"2024-09-12T02:47:52.498955+0000","last_fullsized":"2024-09-12T02:47:52.498955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234907+0000","last_change":"2024-09-12T02:47:39.062615+0000","last_active":"2024-09-12T02:48:19.234907+0000","last_peered":"2024-09-12T02:48:19.234907+0000","last_clean":"2024-09-12T02:48:19.234907+0000","last_became_active":"2024-09-12T02:47:39.062174+0000","last_became_peered":"2024-09-12T02:47:39.062174+0000","last_unstale":"2024-09-12T02:48:19.234907+0000","last_undegraded":"2024-09-12T02:48:19.234907+0000","last_fullsized":"2024-09-12T02:48:19.234907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:10.443649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339237+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:36.339237+0000","last_peered":"2024-09-12T02:48:36.339237+0000","last_clean":"2024-09-12T02:48:36.339237+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:36.339237+0000","last_undegraded":"2024-09-12T02:48:36.339237+0000","last_fullsized":"2024-09-12T02:48:36.339237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404222+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:47:52.404222+0000","last_peered":"2024-09-12T02:47:52.404222+0000","last_clean":"2024-09-12T02:47:52.404222+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:47:52.404222+0000","last_undegraded":"2024-09-12T02:47:52.404222+0000","last_fullsized":"2024-09-12T02:47:52.404222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.133602+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:48:36.133602+0000","last_peered":"2024-09-12T02:48:36.133602+0000","last_clean":"2024-09-12T02:48:36.133602+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:48:36.133602+0000","last_undegraded":"2024-09-12T02:48:36.133602+0000","last_fullsized":"2024-09-12T02:48:36.133602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54396,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":73,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335334+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:36.335334+0000","last_peered":"2024-09-12T02:48:36.335334+0000","last_clean":"2024-09-12T02:48:36.335334+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:36.335334+0000","last_undegraded":"2024-09-12T02:48:36.335334+0000","last_fullsized":"2024-09-12T02:48:36.335334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404520+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:47:52.404520+0000","last_peered":"2024-09-12T02:47:52.404520+0000","last_clean":"2024-09-12T02:47:52.404520+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:47:52.404520+0000","last_undegraded":"2024-09-12T02:47:52.404520+0000","last_fullsized":"2024-09-12T02:47:52.404520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340639+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:36.340639+0000","last_peered":"2024-09-12T02:48:36.340639+0000","last_clean":"2024-09-12T02:48:36.340639+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:36.340639+0000","last_undegraded":"2024-09-12T02:48:36.340639+0000","last_fullsized":"2024-09-12T02:48:36.340639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338081+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:36.338081+0000","last_peered":"2024-09-12T02:48:36.338081+0000","last_clean":"2024-09-12T02:48:36.338081+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:36.338081+0000","last_undegraded":"2024-09-12T02:48:36.338081+0000","last_fullsized":"2024-09-12T02:48:36.338081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403866+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:47:52.403866+0000","last_peered":"2024-09-12T02:47:52.403866+0000","last_clean":"2024-09-12T02:47:52.403866+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:47:52.403866+0000","last_undegraded":"2024-09-12T02:47:52.403866+0000","last_fullsized":"2024-09-12T02:47:52.403866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223376+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:18.223376+0000","last_peered":"2024-09-12T02:48:18.223376+0000","last_clean":"2024-09-12T02:48:18.223376+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:18.223376+0000","last_undegraded":"2024-09-12T02:48:18.223376+0000","last_fullsized":"2024-09-12T02:48:18.223376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335147+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:36.335147+0000","last_peered":"2024-09-12T02:48:36.335147+0000","last_clean":"2024-09-12T02:48:36.335147+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:36.335147+0000","last_undegraded":"2024-09-12T02:48:36.335147+0000","last_fullsized":"2024-09-12T02:48:36.335147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405159+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:47:52.405159+0000","last_peered":"2024-09-12T02:47:52.405159+0000","last_clean":"2024-09-12T02:47:52.405159+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:47:52.405159+0000","last_undegraded":"2024-09-12T02:47:52.405159+0000","last_fullsized":"2024-09-12T02:47:52.405159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:35.127906+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:48:35.127906+0000","last_peered":"2024-09-12T02:48:35.127906+0000","last_clean":"2024-09-12T02:48:35.127906+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:48:35.127906+0000","last_undegraded":"2024-09-12T02:48:35.127906+0000","last_fullsized":"2024-09-12T02:48:35.127906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281217+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:47:52.281217+0000","last_peered":"2024-09-12T02:47:52.281217+0000","last_clean":"2024-09-12T02:47:52.281217+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:47:52.281217+0000","last_undegraded":"2024-09-12T02:47:52.281217+0000","last_fullsized":"2024-09-12T02:47:52.281217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403758+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:47:52.403758+0000","last_peered":"2024-09-12T02:47:52.403758+0000","last_clean":"2024-09-12T02:47:52.403758+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:47:52.403758+0000","last_undegraded":"2024-09-12T02:47:52.403758+0000","last_fullsized":"2024-09-12T02:47:52.403758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335460+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:36.335460+0000","last_peered":"2024-09-12T02:48:36.335460+0000","last_clean":"2024-09-12T02:48:36.335460+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:36.335460+0000","last_undegraded":"2024-09-12T02:48:36.335460+0000","last_fullsized":"2024-09-12T02:48:36.335460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338551+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:36.338551+0000","last_peered":"2024-09-12T02:48:36.338551+0000","last_clean":"2024-09-12T02:48:36.338551+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:36.338551+0000","last_undegraded":"2024-09-12T02:48:36.338551+0000","last_fullsized":"2024-09-12T02:48:36.338551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141988+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:47:51.141988+0000","last_peered":"2024-09-12T02:47:51.141988+0000","last_clean":"2024-09-12T02:47:51.141988+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:47:51.141988+0000","last_undegraded":"2024-09-12T02:47:51.141988+0000","last_fullsized":"2024-09-12T02:47:51.141988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281131+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:47:52.281131+0000","last_peered":"2024-09-12T02:47:52.281131+0000","last_clean":"2024-09-12T02:47:52.281131+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:47:52.281131+0000","last_undegraded":"2024-09-12T02:47:52.281131+0000","last_fullsized":"2024-09-12T02:47:52.281131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234186+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:19.234186+0000","last_peered":"2024-09-12T02:48:19.234186+0000","last_clean":"2024-09-12T02:48:19.234186+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:19.234186+0000","last_undegraded":"2024-09-12T02:48:19.234186+0000","last_fullsized":"2024-09-12T02:48:19.234186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234039+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:19.234039+0000","last_peered":"2024-09-12T02:48:19.234039+0000","last_clean":"2024-09-12T02:48:19.234039+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:19.234039+0000","last_undegraded":"2024-09-12T02:48:19.234039+0000","last_fullsized":"2024-09-12T02:48:19.234039+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338692+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:36.338692+0000","last_peered":"2024-09-12T02:48:36.338692+0000","last_clean":"2024-09-12T02:48:36.338692+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:36.338692+0000","last_undegraded":"2024-09-12T02:48:36.338692+0000","last_fullsized":"2024-09-12T02:48:36.338692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403756+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:47:52.403756+0000","last_peered":"2024-09-12T02:47:52.403756+0000","last_clean":"2024-09-12T02:47:52.403756+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:47:52.403756+0000","last_undegraded":"2024-09-12T02:47:52.403756+0000","last_fullsized":"2024-09-12T02:47:52.403756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338792+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:48:36.338792+0000","last_peered":"2024-09-12T02:48:36.338792+0000","last_clean":"2024-09-12T02:48:36.338792+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:48:36.338792+0000","last_undegraded":"2024-09-12T02:48:36.338792+0000","last_fullsized":"2024-09-12T02:48:36.338792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404036+0000","last_change":"2024-09-12T02:47:41.114331+0000","last_active":"2024-09-12T02:47:52.404036+0000","last_peered":"2024-09-12T02:47:52.404036+0000","last_clean":"2024-09-12T02:47:52.404036+0000","last_became_active":"2024-09-12T02:47:41.113977+0000","last_became_peered":"2024-09-12T02:47:41.113977+0000","last_unstale":"2024-09-12T02:47:52.404036+0000","last_undegraded":"2024-09-12T02:47:52.404036+0000","last_fullsized":"2024-09-12T02:47:52.404036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:41.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403828+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:47:52.403828+0000","last_peered":"2024-09-12T02:47:52.403828+0000","last_clean":"2024-09-12T02:47:52.403828+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:47:52.403828+0000","last_undegraded":"2024-09-12T02:47:52.403828+0000","last_fullsized":"2024-09-12T02:47:52.403828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141425+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:47:51.141425+0000","last_peered":"2024-09-12T02:47:51.141425+0000","last_clean":"2024-09-12T02:47:51.141425+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:47:51.141425+0000","last_undegraded":"2024-09-12T02:47:51.141425+0000","last_fullsized":"2024-09-12T02:47:51.141425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403665+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:47:52.403665+0000","last_peered":"2024-09-12T02:47:52.403665+0000","last_clean":"2024-09-12T02:47:52.403665+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:47:52.403665+0000","last_undegraded":"2024-09-12T02:47:52.403665+0000","last_fullsized":"2024-09-12T02:47:52.403665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403447+0000","last_change":"2024-09-12T02:47:41.119469+0000","last_active":"2024-09-12T02:47:52.403447+0000","last_peered":"2024-09-12T02:47:52.403447+0000","last_clean":"2024-09-12T02:47:52.403447+0000","last_became_active":"2024-09-12T02:47:41.119120+0000","last_became_peered":"2024-09-12T02:47:41.119120+0000","last_unstale":"2024-09-12T02:47:52.403447+0000","last_undegraded":"2024-09-12T02:47:52.403447+0000","last_fullsized":"2024-09-12T02:47:52.403447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:27.297002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403525+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:47:52.403525+0000","last_peered":"2024-09-12T02:47:52.403525+0000","last_clean":"2024-09-12T02:47:52.403525+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:47:52.403525+0000","last_undegraded":"2024-09-12T02:47:52.403525+0000","last_fullsized":"2024-09-12T02:47:52.403525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499329+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:47:52.499329+0000","last_peered":"2024-09-12T02:47:52.499329+0000","last_clean":"2024-09-12T02:47:52.499329+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:47:52.499329+0000","last_undegraded":"2024-09-12T02:47:52.499329+0000","last_fullsized":"2024-09-12T02:47:52.499329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141883+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:47:51.141883+0000","last_peered":"2024-09-12T02:47:51.141883+0000","last_clean":"2024-09-12T02:47:51.141883+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:47:51.141883+0000","last_undegraded":"2024-09-12T02:47:51.141883+0000","last_fullsized":"2024-09-12T02:47:51.141883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403901+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:47:52.403901+0000","last_peered":"2024-09-12T02:47:52.403901+0000","last_clean":"2024-09-12T02:47:52.403901+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:47:52.403901+0000","last_undegraded":"2024-09-12T02:47:52.403901+0000","last_fullsized":"2024-09-12T02:47:52.403901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281451+0000","last_change":"2024-09-12T02:47:41.118766+0000","last_active":"2024-09-12T02:47:52.281451+0000","last_peered":"2024-09-12T02:47:52.281451+0000","last_clean":"2024-09-12T02:47:52.281451+0000","last_became_active":"2024-09-12T02:47:41.118459+0000","last_became_peered":"2024-09-12T02:47:41.118459+0000","last_unstale":"2024-09-12T02:47:52.281451+0000","last_undegraded":"2024-09-12T02:47:52.281451+0000","last_fullsized":"2024-09-12T02:47:52.281451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:12.983552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338261+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:36.338261+0000","last_peered":"2024-09-12T02:48:36.338261+0000","last_clean":"2024-09-12T02:48:36.338261+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:36.338261+0000","last_undegraded":"2024-09-12T02:48:36.338261+0000","last_fullsized":"2024-09-12T02:48:36.338261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281304+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:47:52.281304+0000","last_peered":"2024-09-12T02:47:52.281304+0000","last_clean":"2024-09-12T02:47:52.281304+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:47:52.281304+0000","last_undegraded":"2024-09-12T02:47:52.281304+0000","last_fullsized":"2024-09-12T02:47:52.281304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404536+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:47:52.404536+0000","last_peered":"2024-09-12T02:47:52.404536+0000","last_clean":"2024-09-12T02:47:52.404536+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:47:52.404536+0000","last_undegraded":"2024-09-12T02:47:52.404536+0000","last_fullsized":"2024-09-12T02:47:52.404536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499229+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:47:52.499229+0000","last_peered":"2024-09-12T02:47:52.499229+0000","last_clean":"2024-09-12T02:47:52.499229+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:47:52.499229+0000","last_undegraded":"2024-09-12T02:47:52.499229+0000","last_fullsized":"2024-09-12T02:47:52.499229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141687+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:47:51.141687+0000","last_peered":"2024-09-12T02:47:51.141687+0000","last_clean":"2024-09-12T02:47:51.141687+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:47:51.141687+0000","last_undegraded":"2024-09-12T02:47:51.141687+0000","last_fullsized":"2024-09-12T02:47:51.141687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403866+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:47:52.403866+0000","last_peered":"2024-09-12T02:47:52.403866+0000","last_clean":"2024-09-12T02:47:52.403866+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:47:52.403866+0000","last_undegraded":"2024-09-12T02:47:52.403866+0000","last_fullsized":"2024-09-12T02:47:52.403866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336259+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:36.336259+0000","last_peered":"2024-09-12T02:48:36.336259+0000","last_clean":"2024-09-12T02:48:36.336259+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:36.336259+0000","last_undegraded":"2024-09-12T02:48:36.336259+0000","last_fullsized":"2024-09-12T02:48:36.336259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.142126+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:47:51.142126+0000","last_peered":"2024-09-12T02:47:51.142126+0000","last_clean":"2024-09-12T02:47:51.142126+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:47:51.142126+0000","last_undegraded":"2024-09-12T02:47:51.142126+0000","last_fullsized":"2024-09-12T02:47:51.142126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403793+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:47:52.403793+0000","last_peered":"2024-09-12T02:47:52.403793+0000","last_clean":"2024-09-12T02:47:52.403793+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:47:52.403793+0000","last_undegraded":"2024-09-12T02:47:52.403793+0000","last_fullsized":"2024-09-12T02:47:52.403793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403453+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:47:52.403453+0000","last_peered":"2024-09-12T02:47:52.403453+0000","last_clean":"2024-09-12T02:47:52.403453+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:47:52.403453+0000","last_undegraded":"2024-09-12T02:47:52.403453+0000","last_fullsized":"2024-09-12T02:47:52.403453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403460+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:47:52.403460+0000","last_peered":"2024-09-12T02:47:52.403460+0000","last_clean":"2024-09-12T02:47:52.403460+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:47:52.403460+0000","last_undegraded":"2024-09-12T02:47:52.403460+0000","last_fullsized":"2024-09-12T02:47:52.403460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.499055+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:47:52.499055+0000","last_peered":"2024-09-12T02:47:52.499055+0000","last_clean":"2024-09-12T02:47:52.499055+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:47:52.499055+0000","last_undegraded":"2024-09-12T02:47:52.499055+0000","last_fullsized":"2024-09-12T02:47:52.499055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338571+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:36.338571+0000","last_peered":"2024-09-12T02:48:36.338571+0000","last_clean":"2024-09-12T02:48:36.338571+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:36.338571+0000","last_undegraded":"2024-09-12T02:48:36.338571+0000","last_fullsized":"2024-09-12T02:48:36.338571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405974+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:47:52.405974+0000","last_peered":"2024-09-12T02:47:52.405974+0000","last_clean":"2024-09-12T02:47:52.405974+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:47:52.405974+0000","last_undegraded":"2024-09-12T02:47:52.405974+0000","last_fullsized":"2024-09-12T02:47:52.405974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141982+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:47:51.141982+0000","last_peered":"2024-09-12T02:47:51.141982+0000","last_clean":"2024-09-12T02:47:51.141982+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:47:51.141982+0000","last_undegraded":"2024-09-12T02:47:51.141982+0000","last_fullsized":"2024-09-12T02:47:51.141982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.405199+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:47:52.405199+0000","last_peered":"2024-09-12T02:47:52.405199+0000","last_clean":"2024-09-12T02:47:52.405199+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:47:52.405199+0000","last_undegraded":"2024-09-12T02:47:52.405199+0000","last_fullsized":"2024-09-12T02:47:52.405199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281004+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:47:52.281004+0000","last_peered":"2024-09-12T02:47:52.281004+0000","last_clean":"2024-09-12T02:47:52.281004+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:47:52.281004+0000","last_undegraded":"2024-09-12T02:47:52.281004+0000","last_fullsized":"2024-09-12T02:47:52.281004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.404645+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:47:52.404645+0000","last_peered":"2024-09-12T02:47:52.404645+0000","last_clean":"2024-09-12T02:47:52.404645+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:47:52.404645+0000","last_undegraded":"2024-09-12T02:47:52.404645+0000","last_fullsized":"2024-09-12T02:47:52.404645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141621+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:47:51.141621+0000","last_peered":"2024-09-12T02:47:51.141621+0000","last_clean":"2024-09-12T02:47:51.141621+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:47:51.141621+0000","last_undegraded":"2024-09-12T02:47:51.141621+0000","last_fullsized":"2024-09-12T02:47:51.141621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403861+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:47:52.403861+0000","last_peered":"2024-09-12T02:47:52.403861+0000","last_clean":"2024-09-12T02:47:52.403861+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:47:52.403861+0000","last_undegraded":"2024-09-12T02:47:52.403861+0000","last_fullsized":"2024-09-12T02:47:52.403861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338563+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:36.338563+0000","last_peered":"2024-09-12T02:48:36.338563+0000","last_clean":"2024-09-12T02:48:36.338563+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:36.338563+0000","last_undegraded":"2024-09-12T02:48:36.338563+0000","last_fullsized":"2024-09-12T02:48:36.338563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403725+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:47:52.403725+0000","last_peered":"2024-09-12T02:47:52.403725+0000","last_clean":"2024-09-12T02:47:52.403725+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:47:52.403725+0000","last_undegraded":"2024-09-12T02:47:52.403725+0000","last_fullsized":"2024-09-12T02:47:52.403725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.403733+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:47:52.403733+0000","last_peered":"2024-09-12T02:47:52.403733+0000","last_clean":"2024-09-12T02:47:52.403733+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:47:52.403733+0000","last_undegraded":"2024-09-12T02:47:52.403733+0000","last_fullsized":"2024-09-12T02:47:52.403733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.142059+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:47:51.142059+0000","last_peered":"2024-09-12T02:47:51.142059+0000","last_clean":"2024-09-12T02:47:51.142059+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:47:51.142059+0000","last_undegraded":"2024-09-12T02:47:51.142059+0000","last_fullsized":"2024-09-12T02:47:51.142059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58399,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":127,"num_read_kb":124,"num_write":164,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":482479,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1124143,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":486811,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":482635,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1078546,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":477891,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1083716,"data_compressed":56827,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":978944,"data_stored":492123,"data_compressed":20581,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":478085,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":533940,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":532197,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":479279,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":55974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:48:38.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:48:39.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:39 smithi145 ceph-mon[34674]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:48:39.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:39.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:39 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:39 smithi196 ceph-mon[34587]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:48:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:39 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:39.747 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:39 smithi046 ceph-mon[30951]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:48:39.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:39.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:39 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 -- 172.21.15.46:0/228860997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c100b20 msgr2=0x7fd65c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 --2- 172.21.15.46:0/228860997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c100b20 0x7fd65c100f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02a410 comp rx=0 tx=0).stop 2024-09-12T02:48:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 -- 172.21.15.46:0/228860997 shutdown_connections 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 --2- 172.21.15.46:0/228860997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c10e500 0x7fd65c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 --2- 172.21.15.46:0/228860997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 0x7fd65c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 --2- 172.21.15.46:0/228860997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c100b20 0x7fd65c100f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 -- 172.21.15.46:0/228860997 >> 172.21.15.46:0/228860997 conn(0x7fd65c0fc7d0 msgr2=0x7fd65c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 -- 172.21.15.46:0/228860997 shutdown_connections 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 -- 172.21.15.46:0/228860997 wait complete. 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.166+0000 7fd660994640 1 Processor -- start 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 -- start start 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c100b20 0x7fd65c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 0x7fd65c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c1141f0 con 0x7fd65c10e500 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c114070 con 0x7fd65c101440 2024-09-12T02:48:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd660994640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c114370 con 0x7fd65c100b20 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52438/0 (socket says 172.21.15.46:52438) 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 -- 172.21.15.46:0/1809103792 learned_addr learned my addr 172.21.15.46:0/1809103792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65a59c640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c100b20 0x7fd65c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd653fff640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 0x7fd65c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 -- 172.21.15.46:0/1809103792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c100b20 msgr2=0x7fd65c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c100b20 0x7fd65c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 -- 172.21.15.46:0/1809103792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 msgr2=0x7fd65c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 0x7fd65c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 -- 172.21.15.46:0/1809103792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c19f6c0 con 0x7fd65c10e500 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.167+0000 7fd65ad9d640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd65400ccb0 tx=0x7fd65400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.168+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65400f450 con 0x7fd65c10e500 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.168+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd65400faa0 con 0x7fd65c10e500 2024-09-12T02:48:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.168+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c19f9b0 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.168+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654018b10 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.168+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c1a7290 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.169+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c06a850 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.170+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd65400f5f0 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.171+0000 7fd6537fe640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 0x7fd6480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.171+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd6540cfc70 con 0x7fd65c10e500 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.171+0000 7fd65a59c640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 0x7fd6480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.171+0000 7fd65a59c640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 0x7fd6480a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005ab0 tx=0x7fd64c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.179+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd654014030 con 0x7fd65c10e500 2024-09-12T02:48:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.371+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd65c062630 con 0x7fd6480a2c90 2024-09-12T02:48:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.381+0000 7fd6537fe640 1 -- 172.21.15.46:0/1809103792 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340562 (secure 0 0 0) 0x7fd65c062630 con 0x7fd6480a2c90 2024-09-12T02:48:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:48:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 msgr2=0x7fd6480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 0x7fd6480a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005ab0 tx=0x7fd64c005a20 comp rx=0 tx=0).stop 2024-09-12T02:48:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 msgr2=0x7fd65c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd65400ccb0 tx=0x7fd65400a620 comp rx=0 tx=0).stop 2024-09-12T02:48:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 shutdown_connections 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c10e500 0x7fd65c19efe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd65c101440 0x7fd65c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd6480a2c90 0x7fd6480a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 --2- 172.21.15.46:0/1809103792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd65c100b20 0x7fd65c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 >> 172.21.15.46:0/1809103792 conn(0x7fd65c0fc7d0 msgr2=0x7fd65c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 shutdown_connections 2024-09-12T02:48:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:40.385+0000 7fd660994640 1 -- 172.21.15.46:0/1809103792 wait complete. 2024-09-12T02:48:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:40 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:40 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:40 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:41.110 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-12T02:48:40.273635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649001,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":268,"num_read_kb":248,"num_write":347,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":425,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369500,"kb_used_data":11884,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572324,"statfs":{"total":1151940427776,"available":1151562059776,"internally_reserved":0,"allocated":12169216,"data_stored":7854269,"data_compressed":361699,"data_compressed_allocated":487424,"data_compressed_original":2473984,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":31,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336042+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:36.336042+0000","last_peered":"2024-09-12T02:48:36.336042+0000","last_clean":"2024-09-12T02:48:36.336042+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:36.336042+0000","last_undegraded":"2024-09-12T02:48:36.336042+0000","last_fullsized":"2024-09-12T02:48:36.336042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281137+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:47:52.281137+0000","last_peered":"2024-09-12T02:47:52.281137+0000","last_clean":"2024-09-12T02:47:52.281137+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:47:52.281137+0000","last_undegraded":"2024-09-12T02:47:52.281137+0000","last_fullsized":"2024-09-12T02:47:52.281137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343048+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:48:37.343048+0000","last_peered":"2024-09-12T02:48:37.343048+0000","last_clean":"2024-09-12T02:48:37.343048+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:48:37.343048+0000","last_undegraded":"2024-09-12T02:48:37.343048+0000","last_fullsized":"2024-09-12T02:48:37.343048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281434+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:47:52.281434+0000","last_peered":"2024-09-12T02:47:52.281434+0000","last_clean":"2024-09-12T02:47:52.281434+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:47:52.281434+0000","last_undegraded":"2024-09-12T02:47:52.281434+0000","last_fullsized":"2024-09-12T02:47:52.281434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341105+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:48:37.341105+0000","last_peered":"2024-09-12T02:48:37.341105+0000","last_clean":"2024-09-12T02:48:37.341105+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:48:37.341105+0000","last_undegraded":"2024-09-12T02:48:37.341105+0000","last_fullsized":"2024-09-12T02:48:37.341105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.337863+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:36.337863+0000","last_peered":"2024-09-12T02:48:36.337863+0000","last_clean":"2024-09-12T02:48:36.337863+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:36.337863+0000","last_undegraded":"2024-09-12T02:48:36.337863+0000","last_fullsized":"2024-09-12T02:48:36.337863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341138+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:37.341138+0000","last_peered":"2024-09-12T02:48:37.341138+0000","last_clean":"2024-09-12T02:48:37.341138+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:37.341138+0000","last_undegraded":"2024-09-12T02:48:37.341138+0000","last_fullsized":"2024-09-12T02:48:37.341138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339509+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:48:37.339509+0000","last_peered":"2024-09-12T02:48:37.339509+0000","last_clean":"2024-09-12T02:48:37.339509+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:48:37.339509+0000","last_undegraded":"2024-09-12T02:48:37.339509+0000","last_fullsized":"2024-09-12T02:48:37.339509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338890+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:36.338890+0000","last_peered":"2024-09-12T02:48:36.338890+0000","last_clean":"2024-09-12T02:48:36.338890+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:36.338890+0000","last_undegraded":"2024-09-12T02:48:36.338890+0000","last_fullsized":"2024-09-12T02:48:36.338890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340619+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:48:37.340619+0000","last_peered":"2024-09-12T02:48:37.340619+0000","last_clean":"2024-09-12T02:48:37.340619+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:48:37.340619+0000","last_undegraded":"2024-09-12T02:48:37.340619+0000","last_fullsized":"2024-09-12T02:48:37.340619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340445+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:36.340445+0000","last_peered":"2024-09-12T02:48:36.340445+0000","last_clean":"2024-09-12T02:48:36.340445+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:36.340445+0000","last_undegraded":"2024-09-12T02:48:36.340445+0000","last_fullsized":"2024-09-12T02:48:36.340445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338032+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:36.338032+0000","last_peered":"2024-09-12T02:48:36.338032+0000","last_clean":"2024-09-12T02:48:36.338032+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:36.338032+0000","last_undegraded":"2024-09-12T02:48:36.338032+0000","last_fullsized":"2024-09-12T02:48:36.338032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586858+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:48:37.586858+0000","last_peered":"2024-09-12T02:48:37.586858+0000","last_clean":"2024-09-12T02:48:37.586858+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:48:37.586858+0000","last_undegraded":"2024-09-12T02:48:37.586858+0000","last_fullsized":"2024-09-12T02:48:37.586858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339937+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:37.339937+0000","last_peered":"2024-09-12T02:48:37.339937+0000","last_clean":"2024-09-12T02:48:37.339937+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:37.339937+0000","last_undegraded":"2024-09-12T02:48:37.339937+0000","last_fullsized":"2024-09-12T02:48:37.339937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338314+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:36.338314+0000","last_peered":"2024-09-12T02:48:36.338314+0000","last_clean":"2024-09-12T02:48:36.338314+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:36.338314+0000","last_undegraded":"2024-09-12T02:48:36.338314+0000","last_fullsized":"2024-09-12T02:48:36.338314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280980+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:47:52.280980+0000","last_peered":"2024-09-12T02:47:52.280980+0000","last_clean":"2024-09-12T02:47:52.280980+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:47:52.280980+0000","last_undegraded":"2024-09-12T02:47:52.280980+0000","last_fullsized":"2024-09-12T02:47:52.280980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343983+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:48:37.343983+0000","last_peered":"2024-09-12T02:48:37.343983+0000","last_clean":"2024-09-12T02:48:37.343983+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:48:37.343983+0000","last_undegraded":"2024-09-12T02:48:37.343983+0000","last_fullsized":"2024-09-12T02:48:37.343983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338333+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:36.338333+0000","last_peered":"2024-09-12T02:48:36.338333+0000","last_clean":"2024-09-12T02:48:36.338333+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:36.338333+0000","last_undegraded":"2024-09-12T02:48:36.338333+0000","last_fullsized":"2024-09-12T02:48:36.338333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340770+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:48:37.340770+0000","last_peered":"2024-09-12T02:48:37.340770+0000","last_clean":"2024-09-12T02:48:37.340770+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:48:37.340770+0000","last_undegraded":"2024-09-12T02:48:37.340770+0000","last_fullsized":"2024-09-12T02:48:37.340770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:18.223309+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:18.223309+0000","last_peered":"2024-09-12T02:48:18.223309+0000","last_clean":"2024-09-12T02:48:18.223309+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:18.223309+0000","last_undegraded":"2024-09-12T02:48:18.223309+0000","last_fullsized":"2024-09-12T02:48:18.223309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335865+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:36.335865+0000","last_peered":"2024-09-12T02:48:36.335865+0000","last_clean":"2024-09-12T02:48:36.335865+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:36.335865+0000","last_undegraded":"2024-09-12T02:48:36.335865+0000","last_fullsized":"2024-09-12T02:48:36.335865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339398+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:48:37.339398+0000","last_peered":"2024-09-12T02:48:37.339398+0000","last_clean":"2024-09-12T02:48:37.339398+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:48:37.339398+0000","last_undegraded":"2024-09-12T02:48:37.339398+0000","last_fullsized":"2024-09-12T02:48:37.339398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339366+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:37.339366+0000","last_peered":"2024-09-12T02:48:37.339366+0000","last_clean":"2024-09-12T02:48:37.339366+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:37.339366+0000","last_undegraded":"2024-09-12T02:48:37.339366+0000","last_fullsized":"2024-09-12T02:48:37.339366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-09-12T02:47:52.404393+0000","last_change":"2024-09-12T02:48:36.338444+0000","last_active":"2024-09-12T02:48:36.338444+0000","last_peered":"2024-09-12T02:47:52.404393+0000","last_clean":"2024-09-12T02:47:52.404393+0000","last_became_active":"2024-09-12T02:47:48.158833+0000","last_became_peered":"2024-09-12T02:47:48.158833+0000","last_unstale":"2024-09-12T02:47:52.404393+0000","last_undegraded":"2024-09-12T02:47:52.404393+0000","last_fullsized":"2024-09-12T02:47:52.404393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341047+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:48:37.341047+0000","last_peered":"2024-09-12T02:48:37.341047+0000","last_clean":"2024-09-12T02:48:37.341047+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:48:37.341047+0000","last_undegraded":"2024-09-12T02:48:37.341047+0000","last_fullsized":"2024-09-12T02:48:37.341047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340605+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:36.340605+0000","last_peered":"2024-09-12T02:48:36.340605+0000","last_clean":"2024-09-12T02:48:36.340605+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:36.340605+0000","last_undegraded":"2024-09-12T02:48:36.340605+0000","last_fullsized":"2024-09-12T02:48:36.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343184+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:48:37.343184+0000","last_peered":"2024-09-12T02:48:37.343184+0000","last_clean":"2024-09-12T02:48:37.343184+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:48:37.343184+0000","last_undegraded":"2024-09-12T02:48:37.343184+0000","last_fullsized":"2024-09-12T02:48:37.343184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281499+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:47:52.281499+0000","last_peered":"2024-09-12T02:47:52.281499+0000","last_clean":"2024-09-12T02:47:52.281499+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:47:52.281499+0000","last_undegraded":"2024-09-12T02:47:52.281499+0000","last_fullsized":"2024-09-12T02:47:52.281499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336358+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:36.336358+0000","last_peered":"2024-09-12T02:48:36.336358+0000","last_clean":"2024-09-12T02:48:36.336358+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:36.336358+0000","last_undegraded":"2024-09-12T02:48:36.336358+0000","last_fullsized":"2024-09-12T02:48:36.336358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.280833+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:47:52.280833+0000","last_peered":"2024-09-12T02:47:52.280833+0000","last_clean":"2024-09-12T02:47:52.280833+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:47:52.280833+0000","last_undegraded":"2024-09-12T02:47:52.280833+0000","last_fullsized":"2024-09-12T02:47:52.280833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336705+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:48:37.336705+0000","last_peered":"2024-09-12T02:48:37.336705+0000","last_clean":"2024-09-12T02:48:37.336705+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:48:37.336705+0000","last_undegraded":"2024-09-12T02:48:37.336705+0000","last_fullsized":"2024-09-12T02:48:37.336705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341250+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:37.341250+0000","last_peered":"2024-09-12T02:48:37.341250+0000","last_clean":"2024-09-12T02:48:37.341250+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:37.341250+0000","last_undegraded":"2024-09-12T02:48:37.341250+0000","last_fullsized":"2024-09-12T02:48:37.341250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:48:37.341552+0000","last_peered":"2024-09-12T02:48:37.341552+0000","last_clean":"2024-09-12T02:48:37.341552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:48:37.341552+0000","last_undegraded":"2024-09-12T02:48:37.341552+0000","last_fullsized":"2024-09-12T02:48:37.341552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141747+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:47:51.141747+0000","last_peered":"2024-09-12T02:47:51.141747+0000","last_clean":"2024-09-12T02:47:51.141747+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:47:51.141747+0000","last_undegraded":"2024-09-12T02:47:51.141747+0000","last_fullsized":"2024-09-12T02:47:51.141747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335512+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:36.335512+0000","last_peered":"2024-09-12T02:48:36.335512+0000","last_clean":"2024-09-12T02:48:36.335512+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:36.335512+0000","last_undegraded":"2024-09-12T02:48:36.335512+0000","last_fullsized":"2024-09-12T02:48:36.335512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339541+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:48:37.339541+0000","last_peered":"2024-09-12T02:48:37.339541+0000","last_clean":"2024-09-12T02:48:37.339541+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:48:37.339541+0000","last_undegraded":"2024-09-12T02:48:37.339541+0000","last_fullsized":"2024-09-12T02:48:37.339541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141682+0000","last_change":"2024-09-12T02:47:41.115955+0000","last_active":"2024-09-12T02:47:51.141682+0000","last_peered":"2024-09-12T02:47:51.141682+0000","last_clean":"2024-09-12T02:47:51.141682+0000","last_became_active":"2024-09-12T02:47:41.115604+0000","last_became_peered":"2024-09-12T02:47:41.115604+0000","last_unstale":"2024-09-12T02:47:51.141682+0000","last_undegraded":"2024-09-12T02:47:51.141682+0000","last_fullsized":"2024-09-12T02:47:51.141682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:46.153312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586990+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:48:37.586990+0000","last_peered":"2024-09-12T02:48:37.586990+0000","last_clean":"2024-09-12T02:48:37.586990+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:48:37.586990+0000","last_undegraded":"2024-09-12T02:48:37.586990+0000","last_fullsized":"2024-09-12T02:48:37.586990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336464+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:48:37.336464+0000","last_peered":"2024-09-12T02:48:37.336464+0000","last_clean":"2024-09-12T02:48:37.336464+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:48:37.336464+0000","last_undegraded":"2024-09-12T02:48:37.336464+0000","last_fullsized":"2024-09-12T02:48:37.336464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338309+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:36.338309+0000","last_peered":"2024-09-12T02:48:36.338309+0000","last_clean":"2024-09-12T02:48:36.338309+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:36.338309+0000","last_undegraded":"2024-09-12T02:48:36.338309+0000","last_fullsized":"2024-09-12T02:48:36.338309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:38.132501+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:48:38.132501+0000","last_peered":"2024-09-12T02:48:38.132501+0000","last_clean":"2024-09-12T02:48:38.132501+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:48:38.132501+0000","last_undegraded":"2024-09-12T02:48:38.132501+0000","last_fullsized":"2024-09-12T02:48:38.132501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336272+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:48:37.336272+0000","last_peered":"2024-09-12T02:48:37.336272+0000","last_clean":"2024-09-12T02:48:37.336272+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:48:37.336272+0000","last_undegraded":"2024-09-12T02:48:37.336272+0000","last_fullsized":"2024-09-12T02:48:37.336272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588802+0000","last_change":"2024-09-12T02:48:37.588802+0000","last_active":"2024-09-12T02:48:37.588802+0000","last_peered":"2024-09-12T02:48:37.588802+0000","last_clean":"2024-09-12T02:48:37.588802+0000","last_became_active":"2024-09-12T02:48:37.588326+0000","last_became_peered":"2024-09-12T02:48:37.588326+0000","last_unstale":"2024-09-12T02:48:37.588802+0000","last_undegraded":"2024-09-12T02:48:37.588802+0000","last_fullsized":"2024-09-12T02:48:37.588802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586758+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:37.586758+0000","last_peered":"2024-09-12T02:48:37.586758+0000","last_clean":"2024-09-12T02:48:37.586758+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:37.586758+0000","last_undegraded":"2024-09-12T02:48:37.586758+0000","last_fullsized":"2024-09-12T02:48:37.586758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-09-12T02:47:52.226786+0000","last_change":"2024-09-12T02:48:36.338797+0000","last_active":"2024-09-12T02:48:36.338797+0000","last_peered":"2024-09-12T02:47:52.226786+0000","last_clean":"2024-09-12T02:47:52.226786+0000","last_became_active":"2024-09-12T02:47:48.157488+0000","last_became_peered":"2024-09-12T02:47:48.157488+0000","last_unstale":"2024-09-12T02:47:52.226786+0000","last_undegraded":"2024-09-12T02:47:52.226786+0000","last_fullsized":"2024-09-12T02:47:52.226786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,8,9,10],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281134+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:47:52.281134+0000","last_peered":"2024-09-12T02:47:52.281134+0000","last_clean":"2024-09-12T02:47:52.281134+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:47:52.281134+0000","last_undegraded":"2024-09-12T02:47:52.281134+0000","last_fullsized":"2024-09-12T02:47:52.281134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339118+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:36.339118+0000","last_peered":"2024-09-12T02:48:36.339118+0000","last_clean":"2024-09-12T02:48:36.339118+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:36.339118+0000","last_undegraded":"2024-09-12T02:48:36.339118+0000","last_fullsized":"2024-09-12T02:48:36.339118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586906+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:48:37.586906+0000","last_peered":"2024-09-12T02:48:37.586906+0000","last_clean":"2024-09-12T02:48:37.586906+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:48:37.586906+0000","last_undegraded":"2024-09-12T02:48:37.586906+0000","last_fullsized":"2024-09-12T02:48:37.586906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586973+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:48:37.586973+0000","last_peered":"2024-09-12T02:48:37.586973+0000","last_clean":"2024-09-12T02:48:37.586973+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:48:37.586973+0000","last_undegraded":"2024-09-12T02:48:37.586973+0000","last_fullsized":"2024-09-12T02:48:37.586973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587180+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:48:37.587180+0000","last_peered":"2024-09-12T02:48:37.587180+0000","last_clean":"2024-09-12T02:48:37.587180+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:48:37.587180+0000","last_undegraded":"2024-09-12T02:48:37.587180+0000","last_fullsized":"2024-09-12T02:48:37.587180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343649+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:37.343649+0000","last_peered":"2024-09-12T02:48:37.343649+0000","last_clean":"2024-09-12T02:48:37.343649+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:37.343649+0000","last_undegraded":"2024-09-12T02:48:37.343649+0000","last_fullsized":"2024-09-12T02:48:37.343649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339501+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:48:37.339501+0000","last_peered":"2024-09-12T02:48:37.339501+0000","last_clean":"2024-09-12T02:48:37.339501+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:48:37.339501+0000","last_undegraded":"2024-09-12T02:48:37.339501+0000","last_fullsized":"2024-09-12T02:48:37.339501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587059+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:48:37.587059+0000","last_peered":"2024-09-12T02:48:37.587059+0000","last_clean":"2024-09-12T02:48:37.587059+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:48:37.587059+0000","last_undegraded":"2024-09-12T02:48:37.587059+0000","last_fullsized":"2024-09-12T02:48:37.587059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340757+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:36.340757+0000","last_peered":"2024-09-12T02:48:36.340757+0000","last_clean":"2024-09-12T02:48:36.340757+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:36.340757+0000","last_undegraded":"2024-09-12T02:48:36.340757+0000","last_fullsized":"2024-09-12T02:48:36.340757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.346098+0000","last_change":"2024-09-12T02:48:37.346098+0000","last_active":"2024-09-12T02:48:37.346098+0000","last_peered":"2024-09-12T02:48:37.346098+0000","last_clean":"2024-09-12T02:48:37.346098+0000","last_became_active":"2024-09-12T02:48:37.345696+0000","last_became_peered":"2024-09-12T02:48:37.345696+0000","last_unstale":"2024-09-12T02:48:37.346098+0000","last_undegraded":"2024-09-12T02:48:37.346098+0000","last_fullsized":"2024-09-12T02:48:37.346098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:48.040441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587347+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:48:37.587347+0000","last_peered":"2024-09-12T02:48:37.587347+0000","last_clean":"2024-09-12T02:48:37.587347+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:48:37.587347+0000","last_undegraded":"2024-09-12T02:48:37.587347+0000","last_fullsized":"2024-09-12T02:48:37.587347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336481+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:48:37.336481+0000","last_peered":"2024-09-12T02:48:37.336481+0000","last_clean":"2024-09-12T02:48:37.336481+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:48:37.336481+0000","last_undegraded":"2024-09-12T02:48:37.336481+0000","last_fullsized":"2024-09-12T02:48:37.336481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338132+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:36.338132+0000","last_peered":"2024-09-12T02:48:36.338132+0000","last_clean":"2024-09-12T02:48:36.338132+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:36.338132+0000","last_undegraded":"2024-09-12T02:48:36.338132+0000","last_fullsized":"2024-09-12T02:48:36.338132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341055+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:48:37.341055+0000","last_peered":"2024-09-12T02:48:37.341055+0000","last_clean":"2024-09-12T02:48:37.341055+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:48:37.341055+0000","last_undegraded":"2024-09-12T02:48:37.341055+0000","last_fullsized":"2024-09-12T02:48:37.341055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340312+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:37.340312+0000","last_peered":"2024-09-12T02:48:37.340312+0000","last_clean":"2024-09-12T02:48:37.340312+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:37.340312+0000","last_undegraded":"2024-09-12T02:48:37.340312+0000","last_fullsized":"2024-09-12T02:48:37.340312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338469+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:36.338469+0000","last_peered":"2024-09-12T02:48:36.338469+0000","last_clean":"2024-09-12T02:48:36.338469+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:36.338469+0000","last_undegraded":"2024-09-12T02:48:36.338469+0000","last_fullsized":"2024-09-12T02:48:36.338469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338359+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:36.338359+0000","last_peered":"2024-09-12T02:48:36.338359+0000","last_clean":"2024-09-12T02:48:36.338359+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:36.338359+0000","last_undegraded":"2024-09-12T02:48:36.338359+0000","last_fullsized":"2024-09-12T02:48:36.338359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339395+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:37.339395+0000","last_peered":"2024-09-12T02:48:37.339395+0000","last_clean":"2024-09-12T02:48:37.339395+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:37.339395+0000","last_undegraded":"2024-09-12T02:48:37.339395+0000","last_fullsized":"2024-09-12T02:48:37.339395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.344145+0000","last_change":"2024-09-12T02:48:37.344145+0000","last_active":"2024-09-12T02:48:37.344145+0000","last_peered":"2024-09-12T02:48:37.344145+0000","last_clean":"2024-09-12T02:48:37.344145+0000","last_became_active":"2024-09-12T02:48:37.343694+0000","last_became_peered":"2024-09-12T02:48:37.343694+0000","last_unstale":"2024-09-12T02:48:37.344145+0000","last_undegraded":"2024-09-12T02:48:37.344145+0000","last_fullsized":"2024-09-12T02:48:37.344145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:55.742228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586644+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:48:37.586644+0000","last_peered":"2024-09-12T02:48:37.586644+0000","last_clean":"2024-09-12T02:48:37.586644+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:48:37.586644+0000","last_undegraded":"2024-09-12T02:48:37.586644+0000","last_fullsized":"2024-09-12T02:48:37.586644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338500+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:48:36.338500+0000","last_peered":"2024-09-12T02:48:36.338500+0000","last_clean":"2024-09-12T02:48:36.338500+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:48:36.338500+0000","last_undegraded":"2024-09-12T02:48:36.338500+0000","last_fullsized":"2024-09-12T02:48:36.338500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335854+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:36.335854+0000","last_peered":"2024-09-12T02:48:36.335854+0000","last_clean":"2024-09-12T02:48:36.335854+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:36.335854+0000","last_undegraded":"2024-09-12T02:48:36.335854+0000","last_fullsized":"2024-09-12T02:48:36.335854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587055+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:48:37.587055+0000","last_peered":"2024-09-12T02:48:37.587055+0000","last_clean":"2024-09-12T02:48:37.587055+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:48:37.587055+0000","last_undegraded":"2024-09-12T02:48:37.587055+0000","last_fullsized":"2024-09-12T02:48:37.587055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339970+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:36.339970+0000","last_peered":"2024-09-12T02:48:36.339970+0000","last_clean":"2024-09-12T02:48:36.339970+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:36.339970+0000","last_undegraded":"2024-09-12T02:48:36.339970+0000","last_fullsized":"2024-09-12T02:48:36.339970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341156+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:48:37.341156+0000","last_peered":"2024-09-12T02:48:37.341156+0000","last_clean":"2024-09-12T02:48:37.341156+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:48:37.341156+0000","last_undegraded":"2024-09-12T02:48:37.341156+0000","last_fullsized":"2024-09-12T02:48:37.341156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336133+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:48:37.336133+0000","last_peered":"2024-09-12T02:48:37.336133+0000","last_clean":"2024-09-12T02:48:37.336133+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:48:37.336133+0000","last_undegraded":"2024-09-12T02:48:37.336133+0000","last_fullsized":"2024-09-12T02:48:37.336133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.353056+0000","last_change":"2024-09-12T02:48:37.352674+0000","last_active":"2024-09-12T02:48:37.353056+0000","last_peered":"2024-09-12T02:48:37.353056+0000","last_clean":"2024-09-12T02:48:37.353056+0000","last_became_active":"2024-09-12T02:48:37.345293+0000","last_became_peered":"2024-09-12T02:48:37.345293+0000","last_unstale":"2024-09-12T02:48:37.353056+0000","last_undegraded":"2024-09-12T02:48:37.353056+0000","last_fullsized":"2024-09-12T02:48:37.353056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:00.609598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.339237+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:36.339237+0000","last_peered":"2024-09-12T02:48:36.339237+0000","last_clean":"2024-09-12T02:48:36.339237+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:36.339237+0000","last_undegraded":"2024-09-12T02:48:36.339237+0000","last_fullsized":"2024-09-12T02:48:36.339237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340161+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:48:37.340161+0000","last_peered":"2024-09-12T02:48:37.340161+0000","last_clean":"2024-09-12T02:48:37.340161+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:48:37.340161+0000","last_undegraded":"2024-09-12T02:48:37.340161+0000","last_fullsized":"2024-09-12T02:48:37.340161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.133602+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:48:36.133602+0000","last_peered":"2024-09-12T02:48:36.133602+0000","last_clean":"2024-09-12T02:48:36.133602+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:48:36.133602+0000","last_undegraded":"2024-09-12T02:48:36.133602+0000","last_fullsized":"2024-09-12T02:48:36.133602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54396,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":73,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335334+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:36.335334+0000","last_peered":"2024-09-12T02:48:36.335334+0000","last_clean":"2024-09-12T02:48:36.335334+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:36.335334+0000","last_undegraded":"2024-09-12T02:48:36.335334+0000","last_fullsized":"2024-09-12T02:48:36.335334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339610+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:48:37.339610+0000","last_peered":"2024-09-12T02:48:37.339610+0000","last_clean":"2024-09-12T02:48:37.339610+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:48:37.339610+0000","last_undegraded":"2024-09-12T02:48:37.339610+0000","last_fullsized":"2024-09-12T02:48:37.339610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.340639+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:36.340639+0000","last_peered":"2024-09-12T02:48:36.340639+0000","last_clean":"2024-09-12T02:48:36.340639+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:36.340639+0000","last_undegraded":"2024-09-12T02:48:36.340639+0000","last_fullsized":"2024-09-12T02:48:36.340639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338081+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:36.338081+0000","last_peered":"2024-09-12T02:48:36.338081+0000","last_clean":"2024-09-12T02:48:36.338081+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:36.338081+0000","last_undegraded":"2024-09-12T02:48:36.338081+0000","last_fullsized":"2024-09-12T02:48:36.338081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341169+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:48:37.341169+0000","last_peered":"2024-09-12T02:48:37.341169+0000","last_clean":"2024-09-12T02:48:37.341169+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:48:37.341169+0000","last_undegraded":"2024-09-12T02:48:37.341169+0000","last_fullsized":"2024-09-12T02:48:37.341169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587207+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:37.587207+0000","last_peered":"2024-09-12T02:48:37.587207+0000","last_clean":"2024-09-12T02:48:37.587207+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:37.587207+0000","last_undegraded":"2024-09-12T02:48:37.587207+0000","last_fullsized":"2024-09-12T02:48:37.587207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335147+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:36.335147+0000","last_peered":"2024-09-12T02:48:36.335147+0000","last_clean":"2024-09-12T02:48:36.335147+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:36.335147+0000","last_undegraded":"2024-09-12T02:48:36.335147+0000","last_fullsized":"2024-09-12T02:48:36.335147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340162+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:48:37.340162+0000","last_peered":"2024-09-12T02:48:37.340162+0000","last_clean":"2024-09-12T02:48:37.340162+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:48:37.340162+0000","last_undegraded":"2024-09-12T02:48:37.340162+0000","last_fullsized":"2024-09-12T02:48:37.340162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:35.127906+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:48:35.127906+0000","last_peered":"2024-09-12T02:48:35.127906+0000","last_clean":"2024-09-12T02:48:35.127906+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:48:35.127906+0000","last_undegraded":"2024-09-12T02:48:35.127906+0000","last_fullsized":"2024-09-12T02:48:35.127906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281217+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:47:52.281217+0000","last_peered":"2024-09-12T02:47:52.281217+0000","last_clean":"2024-09-12T02:47:52.281217+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:47:52.281217+0000","last_undegraded":"2024-09-12T02:47:52.281217+0000","last_fullsized":"2024-09-12T02:47:52.281217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340919+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:48:37.340919+0000","last_peered":"2024-09-12T02:48:37.340919+0000","last_clean":"2024-09-12T02:48:37.340919+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:48:37.340919+0000","last_undegraded":"2024-09-12T02:48:37.340919+0000","last_fullsized":"2024-09-12T02:48:37.340919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.335460+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:36.335460+0000","last_peered":"2024-09-12T02:48:36.335460+0000","last_clean":"2024-09-12T02:48:36.335460+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:36.335460+0000","last_undegraded":"2024-09-12T02:48:36.335460+0000","last_fullsized":"2024-09-12T02:48:36.335460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338551+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:36.338551+0000","last_peered":"2024-09-12T02:48:36.338551+0000","last_clean":"2024-09-12T02:48:36.338551+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:36.338551+0000","last_undegraded":"2024-09-12T02:48:36.338551+0000","last_fullsized":"2024-09-12T02:48:36.338551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340797+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:48:37.340797+0000","last_peered":"2024-09-12T02:48:37.340797+0000","last_clean":"2024-09-12T02:48:37.340797+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:48:37.340797+0000","last_undegraded":"2024-09-12T02:48:37.340797+0000","last_fullsized":"2024-09-12T02:48:37.340797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281131+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:47:52.281131+0000","last_peered":"2024-09-12T02:47:52.281131+0000","last_clean":"2024-09-12T02:47:52.281131+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:47:52.281131+0000","last_undegraded":"2024-09-12T02:47:52.281131+0000","last_fullsized":"2024-09-12T02:47:52.281131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336024+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:37.336024+0000","last_peered":"2024-09-12T02:48:37.336024+0000","last_clean":"2024-09-12T02:48:37.336024+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:37.336024+0000","last_undegraded":"2024-09-12T02:48:37.336024+0000","last_fullsized":"2024-09-12T02:48:37.336024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:48:19.234039+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:19.234039+0000","last_peered":"2024-09-12T02:48:19.234039+0000","last_clean":"2024-09-12T02:48:19.234039+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:19.234039+0000","last_undegraded":"2024-09-12T02:48:19.234039+0000","last_fullsized":"2024-09-12T02:48:19.234039+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338692+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:36.338692+0000","last_peered":"2024-09-12T02:48:36.338692+0000","last_clean":"2024-09-12T02:48:36.338692+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:36.338692+0000","last_undegraded":"2024-09-12T02:48:36.338692+0000","last_fullsized":"2024-09-12T02:48:36.338692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343312+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:48:37.343312+0000","last_peered":"2024-09-12T02:48:37.343312+0000","last_clean":"2024-09-12T02:48:37.343312+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:48:37.343312+0000","last_undegraded":"2024-09-12T02:48:37.343312+0000","last_fullsized":"2024-09-12T02:48:37.343312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338792+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:48:36.338792+0000","last_peered":"2024-09-12T02:48:36.338792+0000","last_clean":"2024-09-12T02:48:36.338792+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:48:36.338792+0000","last_undegraded":"2024-09-12T02:48:36.338792+0000","last_fullsized":"2024-09-12T02:48:36.338792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588956+0000","last_change":"2024-09-12T02:48:37.588956+0000","last_active":"2024-09-12T02:48:37.588956+0000","last_peered":"2024-09-12T02:48:37.588956+0000","last_clean":"2024-09-12T02:48:37.588956+0000","last_became_active":"2024-09-12T02:48:37.588561+0000","last_became_peered":"2024-09-12T02:48:37.588561+0000","last_unstale":"2024-09-12T02:48:37.588956+0000","last_undegraded":"2024-09-12T02:48:37.588956+0000","last_fullsized":"2024-09-12T02:48:37.588956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:08.645076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343699+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:48:37.343699+0000","last_peered":"2024-09-12T02:48:37.343699+0000","last_clean":"2024-09-12T02:48:37.343699+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:48:37.343699+0000","last_undegraded":"2024-09-12T02:48:37.343699+0000","last_fullsized":"2024-09-12T02:48:37.343699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340641+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:48:37.340641+0000","last_peered":"2024-09-12T02:48:37.340641+0000","last_clean":"2024-09-12T02:48:37.340641+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:48:37.340641+0000","last_undegraded":"2024-09-12T02:48:37.340641+0000","last_fullsized":"2024-09-12T02:48:37.340641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343329+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:48:37.343329+0000","last_peered":"2024-09-12T02:48:37.343329+0000","last_clean":"2024-09-12T02:48:37.343329+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:48:37.343329+0000","last_undegraded":"2024-09-12T02:48:37.343329+0000","last_fullsized":"2024-09-12T02:48:37.343329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.589056+0000","last_change":"2024-09-12T02:48:37.589056+0000","last_active":"2024-09-12T02:48:37.589056+0000","last_peered":"2024-09-12T02:48:37.589056+0000","last_clean":"2024-09-12T02:48:37.589056+0000","last_became_active":"2024-09-12T02:48:37.588629+0000","last_became_peered":"2024-09-12T02:48:37.588629+0000","last_unstale":"2024-09-12T02:48:37.589056+0000","last_undegraded":"2024-09-12T02:48:37.589056+0000","last_fullsized":"2024-09-12T02:48:37.589056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:03.490341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340249+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:48:37.340249+0000","last_peered":"2024-09-12T02:48:37.340249+0000","last_clean":"2024-09-12T02:48:37.340249+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:48:37.340249+0000","last_undegraded":"2024-09-12T02:48:37.340249+0000","last_fullsized":"2024-09-12T02:48:37.340249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336590+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:48:37.336590+0000","last_peered":"2024-09-12T02:48:37.336590+0000","last_clean":"2024-09-12T02:48:37.336590+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:48:37.336590+0000","last_undegraded":"2024-09-12T02:48:37.336590+0000","last_fullsized":"2024-09-12T02:48:37.336590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341713+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:48:37.341713+0000","last_peered":"2024-09-12T02:48:37.341713+0000","last_clean":"2024-09-12T02:48:37.341713+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:48:37.341713+0000","last_undegraded":"2024-09-12T02:48:37.341713+0000","last_fullsized":"2024-09-12T02:48:37.341713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343987+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:48:37.343987+0000","last_peered":"2024-09-12T02:48:37.343987+0000","last_clean":"2024-09-12T02:48:37.343987+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:48:37.343987+0000","last_undegraded":"2024-09-12T02:48:37.343987+0000","last_fullsized":"2024-09-12T02:48:37.343987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281451+0000","last_change":"2024-09-12T02:47:41.118766+0000","last_active":"2024-09-12T02:47:52.281451+0000","last_peered":"2024-09-12T02:47:52.281451+0000","last_clean":"2024-09-12T02:47:52.281451+0000","last_became_active":"2024-09-12T02:47:41.118459+0000","last_became_peered":"2024-09-12T02:47:41.118459+0000","last_unstale":"2024-09-12T02:47:52.281451+0000","last_undegraded":"2024-09-12T02:47:52.281451+0000","last_fullsized":"2024-09-12T02:47:52.281451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:12.983552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338261+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:36.338261+0000","last_peered":"2024-09-12T02:48:36.338261+0000","last_clean":"2024-09-12T02:48:36.338261+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:36.338261+0000","last_undegraded":"2024-09-12T02:48:36.338261+0000","last_fullsized":"2024-09-12T02:48:36.338261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281304+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:47:52.281304+0000","last_peered":"2024-09-12T02:47:52.281304+0000","last_clean":"2024-09-12T02:47:52.281304+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:47:52.281304+0000","last_undegraded":"2024-09-12T02:47:52.281304+0000","last_fullsized":"2024-09-12T02:47:52.281304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339814+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:48:37.339814+0000","last_peered":"2024-09-12T02:48:37.339814+0000","last_clean":"2024-09-12T02:48:37.339814+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:48:37.339814+0000","last_undegraded":"2024-09-12T02:48:37.339814+0000","last_fullsized":"2024-09-12T02:48:37.339814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336265+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:48:37.336265+0000","last_peered":"2024-09-12T02:48:37.336265+0000","last_clean":"2024-09-12T02:48:37.336265+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:48:37.336265+0000","last_undegraded":"2024-09-12T02:48:37.336265+0000","last_fullsized":"2024-09-12T02:48:37.336265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341263+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:48:37.341263+0000","last_peered":"2024-09-12T02:48:37.341263+0000","last_clean":"2024-09-12T02:48:37.341263+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:48:37.341263+0000","last_undegraded":"2024-09-12T02:48:37.341263+0000","last_fullsized":"2024-09-12T02:48:37.341263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343830+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:48:37.343830+0000","last_peered":"2024-09-12T02:48:37.343830+0000","last_clean":"2024-09-12T02:48:37.343830+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:48:37.343830+0000","last_undegraded":"2024-09-12T02:48:37.343830+0000","last_fullsized":"2024-09-12T02:48:37.343830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.336259+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:36.336259+0000","last_peered":"2024-09-12T02:48:36.336259+0000","last_clean":"2024-09-12T02:48:36.336259+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:36.336259+0000","last_undegraded":"2024-09-12T02:48:36.336259+0000","last_fullsized":"2024-09-12T02:48:36.336259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342161+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:48:37.342161+0000","last_peered":"2024-09-12T02:48:37.342161+0000","last_clean":"2024-09-12T02:48:37.342161+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:48:37.342161+0000","last_undegraded":"2024-09-12T02:48:37.342161+0000","last_fullsized":"2024-09-12T02:48:37.342161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339233+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:48:37.339233+0000","last_peered":"2024-09-12T02:48:37.339233+0000","last_clean":"2024-09-12T02:48:37.339233+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:48:37.339233+0000","last_undegraded":"2024-09-12T02:48:37.339233+0000","last_fullsized":"2024-09-12T02:48:37.339233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339694+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:48:37.339694+0000","last_peered":"2024-09-12T02:48:37.339694+0000","last_clean":"2024-09-12T02:48:37.339694+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:48:37.339694+0000","last_undegraded":"2024-09-12T02:48:37.339694+0000","last_fullsized":"2024-09-12T02:48:37.339694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339992+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:48:37.339992+0000","last_peered":"2024-09-12T02:48:37.339992+0000","last_clean":"2024-09-12T02:48:37.339992+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:48:37.339992+0000","last_undegraded":"2024-09-12T02:48:37.339992+0000","last_fullsized":"2024-09-12T02:48:37.339992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336229+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:48:37.336229+0000","last_peered":"2024-09-12T02:48:37.336229+0000","last_clean":"2024-09-12T02:48:37.336229+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:48:37.336229+0000","last_undegraded":"2024-09-12T02:48:37.336229+0000","last_fullsized":"2024-09-12T02:48:37.336229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338571+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:36.338571+0000","last_peered":"2024-09-12T02:48:36.338571+0000","last_clean":"2024-09-12T02:48:36.338571+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:36.338571+0000","last_undegraded":"2024-09-12T02:48:36.338571+0000","last_fullsized":"2024-09-12T02:48:36.338571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587114+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:48:37.587114+0000","last_peered":"2024-09-12T02:48:37.587114+0000","last_clean":"2024-09-12T02:48:37.587114+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:48:37.587114+0000","last_undegraded":"2024-09-12T02:48:37.587114+0000","last_fullsized":"2024-09-12T02:48:37.587114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141982+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:47:51.141982+0000","last_peered":"2024-09-12T02:47:51.141982+0000","last_clean":"2024-09-12T02:47:51.141982+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:47:51.141982+0000","last_undegraded":"2024-09-12T02:47:51.141982+0000","last_fullsized":"2024-09-12T02:47:51.141982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340009+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:48:37.340009+0000","last_peered":"2024-09-12T02:48:37.340009+0000","last_clean":"2024-09-12T02:48:37.340009+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:48:37.340009+0000","last_undegraded":"2024-09-12T02:48:37.340009+0000","last_fullsized":"2024-09-12T02:48:37.340009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:52.281004+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:47:52.281004+0000","last_peered":"2024-09-12T02:47:52.281004+0000","last_clean":"2024-09-12T02:47:52.281004+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:47:52.281004+0000","last_undegraded":"2024-09-12T02:47:52.281004+0000","last_fullsized":"2024-09-12T02:47:52.281004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586757+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:48:37.586757+0000","last_peered":"2024-09-12T02:48:37.586757+0000","last_clean":"2024-09-12T02:48:37.586757+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:48:37.586757+0000","last_undegraded":"2024-09-12T02:48:37.586757+0000","last_fullsized":"2024-09-12T02:48:37.586757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T02:47:51.141621+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:47:51.141621+0000","last_peered":"2024-09-12T02:47:51.141621+0000","last_clean":"2024-09-12T02:47:51.141621+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:47:51.141621+0000","last_undegraded":"2024-09-12T02:47:51.141621+0000","last_fullsized":"2024-09-12T02:47:51.141621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339808+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:48:37.339808+0000","last_peered":"2024-09-12T02:48:37.339808+0000","last_clean":"2024-09-12T02:48:37.339808+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:48:37.339808+0000","last_undegraded":"2024-09-12T02:48:37.339808+0000","last_fullsized":"2024-09-12T02:48:37.339808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T02:48:36.338563+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:36.338563+0000","last_peered":"2024-09-12T02:48:36.338563+0000","last_clean":"2024-09-12T02:48:36.338563+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:36.338563+0000","last_undegraded":"2024-09-12T02:48:36.338563+0000","last_fullsized":"2024-09-12T02:48:36.338563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340897+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:48:37.340897+0000","last_peered":"2024-09-12T02:48:37.340897+0000","last_clean":"2024-09-12T02:48:37.340897+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:48:37.340897+0000","last_undegraded":"2024-09-12T02:48:37.340897+0000","last_fullsized":"2024-09-12T02:48:37.340897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341040+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:48:37.341040+0000","last_peered":"2024-09-12T02:48:37.341040+0000","last_clean":"2024-09-12T02:48:37.341040+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:48:37.341040+0000","last_undegraded":"2024-09-12T02:48:37.341040+0000","last_fullsized":"2024-09-12T02:48:37.341040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342054+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:48:37.342054+0000","last_peered":"2024-09-12T02:48:37.342054+0000","last_clean":"2024-09-12T02:48:37.342054+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:48:37.342054+0000","last_undegraded":"2024-09-12T02:48:37.342054+0000","last_fullsized":"2024-09-12T02:48:37.342054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58399,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":164,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":183727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":501459,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1124143,"data_compressed":53850,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":486811,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":501581,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1088128,"data_compressed":56827,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":496837,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1044480,"data_stored":1083716,"data_compressed":56827,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":978944,"data_stored":492123,"data_compressed":20581,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":478085,"data_compressed":17604,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":557117,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":555408,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":488861,"data_compressed":20581,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":55974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:48:41.115 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-12T02:48:41.115 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 18, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2024-09-12T02:47:52.404393+0000', 'last_change': '2024-09-12T02:48:36.338444+0000', 'last_active': '2024-09-12T02:48:36.338444+0000', 'last_peered': '2024-09-12T02:47:52.404393+0000', 'last_clean': '2024-09-12T02:47:52.404393+0000', 'last_became_active': '2024-09-12T02:47:48.158833+0000', 'last_became_peered': '2024-09-12T02:47:48.158833+0000', 'last_unstale': '2024-09-12T02:47:52.404393+0000', 'last_undegraded': '2024-09-12T02:47:52.404393+0000', 'last_fullsized': '2024-09-12T02:47:52.404393+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 86, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'last_clean_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 8, 10, 9], 'acting': [4, 8, 10, 9], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 8, 9, 10], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T02:48:41.115 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-12T02:48:41.115 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 18, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2024-09-12T02:47:52.226786+0000', 'last_change': '2024-09-12T02:48:36.338797+0000', 'last_active': '2024-09-12T02:48:36.338797+0000', 'last_peered': '2024-09-12T02:47:52.226786+0000', 'last_clean': '2024-09-12T02:47:52.226786+0000', 'last_became_active': '2024-09-12T02:47:48.157488+0000', 'last_became_peered': '2024-09-12T02:47:48.157488+0000', 'last_unstale': '2024-09-12T02:47:52.226786+0000', 'last_undegraded': '2024-09-12T02:47:52.226786+0000', 'last_fullsized': '2024-09-12T02:47:52.226786+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 86, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'last_clean_scrub_stamp': '2024-09-12T02:47:47.125152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 8, 6, 10], 'acting': [5, 8, 6, 10], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6, 8, 9, 10], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T02:48:41.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-12T02:48:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 -- 172.21.15.46:0/1391754782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4102ef0 msgr2=0x7feef410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 --2- 172.21.15.46:0/1391754782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4102ef0 0x7feef410da40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feeec009ca0 tx=0x7feeec02a950 comp rx=0 tx=0).stop 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 -- 172.21.15.46:0/1391754782 shutdown_connections 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 --2- 172.21.15.46:0/1391754782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4102ef0 0x7feef410da40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 --2- 172.21.15.46:0/1391754782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102570 0x7feef41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 --2- 172.21.15.46:0/1391754782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4100e20 0x7feef4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 -- 172.21.15.46:0/1391754782 >> 172.21.15.46:0/1391754782 conn(0x7feef40fc7d0 msgr2=0x7feef40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 -- 172.21.15.46:0/1391754782 shutdown_connections 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.924+0000 7feefafe7640 1 -- 172.21.15.46:0/1391754782 wait complete. 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 Processor -- start 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 -- start start 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4102570 0x7feef4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102ef0 0x7feef419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feef4110e30 con 0x7feef4100e20 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feef4110cb0 con 0x7feef4102ef0 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feefafe7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feef4110fb0 con 0x7feef4102570 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feef8d83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feef8d83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52458/0 (socket says 172.21.15.46:52458) 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feef8d83640 1 -- 172.21.15.46:0/387300780 learned_addr learned my addr 172.21.15.46:0/387300780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.925+0000 7feef9584640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102ef0 0x7feef419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feeebfff640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4102570 0x7feef4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 -- 172.21.15.46:0/387300780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4102570 msgr2=0x7feef4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4102570 0x7feef4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 -- 172.21.15.46:0/387300780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102ef0 msgr2=0x7feef419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102ef0 0x7feef419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 -- 172.21.15.46:0/387300780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef419d2b0 con 0x7feef4100e20 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feef8d83640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feef000c920 tx=0x7feef000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0007c70 con 0x7feef4100e20 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef0007e10 con 0x7feef4100e20 2024-09-12T02:48:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef419d5a0 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0018820 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.926+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef41a4e80 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.928+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef4101200 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.929+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feef00189c0 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.930+0000 7feee9ffb640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 0x7feedc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.930+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7feef00cf610 con 0x7feef4100e20 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.930+0000 7feeebfff640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 0x7feedc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.930+0000 7feeebfff640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 0x7feedc0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feef40ff0e0 tx=0x7feee4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:41.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:41.934+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feef0014030 con 0x7feef4100e20 2024-09-12T02:48:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.181+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7feef4199660 con 0x7feef4100e20 2024-09-12T02:48:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.181+0000 7feee9ffb640 1 -- 172.21.15.46:0/387300780 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2584 (secure 0 0 0) 0x7feef009c050 con 0x7feef4100e20 2024-09-12T02:48:42.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:42.185 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":435,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1726109208,"num_in_osds":12,"osd_in_since":1726109184,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":649001,"bytes_used":378368000,"bytes_avail":1151562059776,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":4,"recovering_keys_per_sec":1,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"read_bytes_sec":3966,"write_bytes_sec":1791,"read_op_per_sec":4,"write_op_per_sec":1},"fsmap":{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24427},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24446},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24454},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24460},{"filesystem_id":1,"rank":1,"name":"b","status":"up:active","gid":24476},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24478}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.46:8443/"}},"servicemap":{"epoch":28,"modified":"2024-09-12T02:48:30.271165+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 msgr2=0x7feedc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 0x7feedc0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feef40ff0e0 tx=0x7feee4005f90 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 msgr2=0x7feef4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feef000c920 tx=0x7feef000cde0 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 shutdown_connections 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feef4102ef0 0x7feef419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7feedc0a2c10 0x7feedc0a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef4102570 0x7feef4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 --2- 172.21.15.46:0/387300780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feef4100e20 0x7feef4198300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.183+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 >> 172.21.15.46:0/387300780 conn(0x7feef40fc7d0 msgr2=0x7feef406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.184+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 shutdown_connections 2024-09-12T02:48:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:42.184+0000 7feefafe7640 1 -- 172.21.15.46:0/387300780 wait complete. 2024-09-12T02:48:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:42 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/387300780' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:42 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/387300780' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T02:48:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 634 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:48:42.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:48:42.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:48:42.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:42 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/387300780' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T02:48:42.886 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-12T02:48:42.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:48:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:43.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- 172.21.15.46:0/3965036541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00102ef0 msgr2=0x7f5a0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/3965036541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00102ef0 0x7f5a0010da40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009ca0 tx=0x7f59f802a950 comp rx=0 tx=0).stop 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- 172.21.15.46:0/3965036541 shutdown_connections 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/3965036541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00102ef0 0x7f5a0010da40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/3965036541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102570 0x7f5a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/3965036541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00100e20 0x7f5a00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- 172.21.15.46:0/3965036541 >> 172.21.15.46:0/3965036541 conn(0x7f5a000fc7d0 msgr2=0x7f5a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- 172.21.15.46:0/3965036541 shutdown_connections 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- 172.21.15.46:0/3965036541 wait complete. 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 Processor -- start 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.615+0000 7f5a05ddd640 1 -- start start 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00102570 0x7f5a000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102ef0 0x7f5a000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00110e30 con 0x7f5a00100e20 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00110cb0 con 0x7f5a00102ef0 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f5a05ddd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00110fb0 con 0x7f5a00102570 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52478/0 (socket says 172.21.15.46:52478) 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 -- 172.21.15.46:0/69884489 learned_addr learned my addr 172.21.15.46:0/69884489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59feffd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00102570 0x7f5a000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59fffff640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102ef0 0x7f5a000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 -- 172.21.15.46:0/69884489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00102570 msgr2=0x7f5a000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00102570 0x7f5a000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 -- 172.21.15.46:0/69884489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102ef0 msgr2=0x7f5a000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102ef0 0x7f5a000fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 -- 172.21.15.46:0/69884489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a001a6700 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.616+0000 7f59ff7fe640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c920 tx=0x7f59f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.617+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0007c70 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.617+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a001a69f0 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.617+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59f0007e10 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.617+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0018820 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.617+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a001a6f80 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.618+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a00101200 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.619+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f59f00189c0 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.619+0000 7f59fcff9640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 0x7f59e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.620+0000 7f59feffd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 0x7f59e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.620+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f59f00cf990 con 0x7f5a00100e20 2024-09-12T02:48:43.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.620+0000 7f59feffd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 0x7f59e80a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a000fe710 tx=0x7f59f4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.623+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59f0014030 con 0x7f5a00100e20 2024-09-12T02:48:43.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.796+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a00062630 con 0x7f59e80a2c90 2024-09-12T02:48:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.804+0000 7f59fcff9640 1 -- 172.21.15.46:0/69884489 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340622 (secure 0 0 0) 0x7f5a00062630 con 0x7f59e80a2c90 2024-09-12T02:48:43.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:48:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 msgr2=0x7f59e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 0x7f59e80a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a000fe710 tx=0x7f59f4005c50 comp rx=0 tx=0).stop 2024-09-12T02:48:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 msgr2=0x7f5a000fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c920 tx=0x7f59f000cde0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 shutdown_connections 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a00102ef0 0x7f5a000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f59e80a2c90 0x7f59e80a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a00102570 0x7f5a000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 --2- 172.21.15.46:0/69884489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a00100e20 0x7f5a000fcc80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 >> 172.21.15.46:0/69884489 conn(0x7f5a000fc7d0 msgr2=0x7f5a0006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 shutdown_connections 2024-09-12T02:48:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:43.809+0000 7f5a05ddd640 1 -- 172.21.15.46:0/69884489 wait complete. 2024-09-12T02:48:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:44 smithi196 ceph-mon[34587]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:44.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:44 smithi046 ceph-mon[30951]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:44.560 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-12T02:48:42.274351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653266,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":294,"num_read_kb":263,"num_write":350,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369624,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572200,"statfs":{"total":1151940427776,"available":1151561932800,"internally_reserved":0,"allocated":12271616,"data_stored":7934100,"data_compressed":385550,"data_compressed_allocated":520192,"data_compressed_original":2539520,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":46,"num_write":14,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002158"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588213+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:37.588213+0000","last_peered":"2024-09-12T02:48:37.588213+0000","last_clean":"2024-09-12T02:48:37.588213+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:37.588213+0000","last_undegraded":"2024-09-12T02:48:37.588213+0000","last_fullsized":"2024-09-12T02:48:37.588213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339671+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:48:37.339671+0000","last_peered":"2024-09-12T02:48:37.339671+0000","last_clean":"2024-09-12T02:48:37.339671+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:48:37.339671+0000","last_undegraded":"2024-09-12T02:48:37.339671+0000","last_fullsized":"2024-09-12T02:48:37.339671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343048+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:48:37.343048+0000","last_peered":"2024-09-12T02:48:37.343048+0000","last_clean":"2024-09-12T02:48:37.343048+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:48:37.343048+0000","last_undegraded":"2024-09-12T02:48:37.343048+0000","last_fullsized":"2024-09-12T02:48:37.343048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339882+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:48:37.339882+0000","last_peered":"2024-09-12T02:48:37.339882+0000","last_clean":"2024-09-12T02:48:37.339882+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:48:37.339882+0000","last_undegraded":"2024-09-12T02:48:37.339882+0000","last_fullsized":"2024-09-12T02:48:37.339882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341105+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:48:37.341105+0000","last_peered":"2024-09-12T02:48:37.341105+0000","last_clean":"2024-09-12T02:48:37.341105+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:48:37.341105+0000","last_undegraded":"2024-09-12T02:48:37.341105+0000","last_fullsized":"2024-09-12T02:48:37.341105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340351+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:37.340351+0000","last_peered":"2024-09-12T02:48:37.340351+0000","last_clean":"2024-09-12T02:48:37.340351+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:37.340351+0000","last_undegraded":"2024-09-12T02:48:37.340351+0000","last_fullsized":"2024-09-12T02:48:37.340351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341138+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:37.341138+0000","last_peered":"2024-09-12T02:48:37.341138+0000","last_clean":"2024-09-12T02:48:37.341138+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:37.341138+0000","last_undegraded":"2024-09-12T02:48:37.341138+0000","last_fullsized":"2024-09-12T02:48:37.341138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339509+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:48:37.339509+0000","last_peered":"2024-09-12T02:48:37.339509+0000","last_clean":"2024-09-12T02:48:37.339509+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:48:37.339509+0000","last_undegraded":"2024-09-12T02:48:37.339509+0000","last_fullsized":"2024-09-12T02:48:37.339509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341184+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:37.341184+0000","last_peered":"2024-09-12T02:48:37.341184+0000","last_clean":"2024-09-12T02:48:37.341184+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:37.341184+0000","last_undegraded":"2024-09-12T02:48:37.341184+0000","last_fullsized":"2024-09-12T02:48:37.341184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340619+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:48:37.340619+0000","last_peered":"2024-09-12T02:48:37.340619+0000","last_clean":"2024-09-12T02:48:37.340619+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:48:37.340619+0000","last_undegraded":"2024-09-12T02:48:37.340619+0000","last_fullsized":"2024-09-12T02:48:37.340619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338203+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:37.338203+0000","last_peered":"2024-09-12T02:48:37.338203+0000","last_clean":"2024-09-12T02:48:37.338203+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:37.338203+0000","last_undegraded":"2024-09-12T02:48:37.338203+0000","last_fullsized":"2024-09-12T02:48:37.338203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340502+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:37.340502+0000","last_peered":"2024-09-12T02:48:37.340502+0000","last_clean":"2024-09-12T02:48:37.340502+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:37.340502+0000","last_undegraded":"2024-09-12T02:48:37.340502+0000","last_fullsized":"2024-09-12T02:48:37.340502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586858+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:48:37.586858+0000","last_peered":"2024-09-12T02:48:37.586858+0000","last_clean":"2024-09-12T02:48:37.586858+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:48:37.586858+0000","last_undegraded":"2024-09-12T02:48:37.586858+0000","last_fullsized":"2024-09-12T02:48:37.586858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339937+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:37.339937+0000","last_peered":"2024-09-12T02:48:37.339937+0000","last_clean":"2024-09-12T02:48:37.339937+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:37.339937+0000","last_undegraded":"2024-09-12T02:48:37.339937+0000","last_fullsized":"2024-09-12T02:48:37.339937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340671+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:37.340671+0000","last_peered":"2024-09-12T02:48:37.340671+0000","last_clean":"2024-09-12T02:48:37.340671+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:37.340671+0000","last_undegraded":"2024-09-12T02:48:37.340671+0000","last_fullsized":"2024-09-12T02:48:37.340671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339210+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:48:37.339210+0000","last_peered":"2024-09-12T02:48:37.339210+0000","last_clean":"2024-09-12T02:48:37.339210+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:48:37.339210+0000","last_undegraded":"2024-09-12T02:48:37.339210+0000","last_fullsized":"2024-09-12T02:48:37.339210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343983+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:48:37.343983+0000","last_peered":"2024-09-12T02:48:37.343983+0000","last_clean":"2024-09-12T02:48:37.343983+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:48:37.343983+0000","last_undegraded":"2024-09-12T02:48:37.343983+0000","last_fullsized":"2024-09-12T02:48:37.343983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341063+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:37.341063+0000","last_peered":"2024-09-12T02:48:37.341063+0000","last_clean":"2024-09-12T02:48:37.341063+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:37.341063+0000","last_undegraded":"2024-09-12T02:48:37.341063+0000","last_fullsized":"2024-09-12T02:48:37.341063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340770+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:48:37.340770+0000","last_peered":"2024-09-12T02:48:37.340770+0000","last_clean":"2024-09-12T02:48:37.340770+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:48:37.340770+0000","last_undegraded":"2024-09-12T02:48:37.340770+0000","last_fullsized":"2024-09-12T02:48:37.340770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339382+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:37.339382+0000","last_peered":"2024-09-12T02:48:37.339382+0000","last_clean":"2024-09-12T02:48:37.339382+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:37.339382+0000","last_undegraded":"2024-09-12T02:48:37.339382+0000","last_fullsized":"2024-09-12T02:48:37.339382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588035+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:37.588035+0000","last_peered":"2024-09-12T02:48:37.588035+0000","last_clean":"2024-09-12T02:48:37.588035+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:37.588035+0000","last_undegraded":"2024-09-12T02:48:37.588035+0000","last_fullsized":"2024-09-12T02:48:37.588035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339398+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:48:37.339398+0000","last_peered":"2024-09-12T02:48:37.339398+0000","last_clean":"2024-09-12T02:48:37.339398+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:48:37.339398+0000","last_undegraded":"2024-09-12T02:48:37.339398+0000","last_fullsized":"2024-09-12T02:48:37.339398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339366+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:37.339366+0000","last_peered":"2024-09-12T02:48:37.339366+0000","last_clean":"2024-09-12T02:48:37.339366+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:37.339366+0000","last_undegraded":"2024-09-12T02:48:37.339366+0000","last_fullsized":"2024-09-12T02:48:37.339366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.590486+0000","last_change":"2024-09-12T02:48:37.590486+0000","last_active":"2024-09-12T02:48:37.590486+0000","last_peered":"2024-09-12T02:48:37.590486+0000","last_clean":"2024-09-12T02:48:37.590486+0000","last_became_active":"2024-09-12T02:48:37.589930+0000","last_became_peered":"2024-09-12T02:48:37.589930+0000","last_unstale":"2024-09-12T02:48:37.590486+0000","last_undegraded":"2024-09-12T02:48:37.590486+0000","last_fullsized":"2024-09-12T02:48:37.590486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:42.323965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341047+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:48:37.341047+0000","last_peered":"2024-09-12T02:48:37.341047+0000","last_clean":"2024-09-12T02:48:37.341047+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:48:37.341047+0000","last_undegraded":"2024-09-12T02:48:37.341047+0000","last_fullsized":"2024-09-12T02:48:37.341047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340953+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:37.340953+0000","last_peered":"2024-09-12T02:48:37.340953+0000","last_clean":"2024-09-12T02:48:37.340953+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:37.340953+0000","last_undegraded":"2024-09-12T02:48:37.340953+0000","last_fullsized":"2024-09-12T02:48:37.340953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343184+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:48:37.343184+0000","last_peered":"2024-09-12T02:48:37.343184+0000","last_clean":"2024-09-12T02:48:37.343184+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:48:37.343184+0000","last_undegraded":"2024-09-12T02:48:37.343184+0000","last_fullsized":"2024-09-12T02:48:37.343184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339781+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:48:37.339781+0000","last_peered":"2024-09-12T02:48:37.339781+0000","last_clean":"2024-09-12T02:48:37.339781+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:48:37.339781+0000","last_undegraded":"2024-09-12T02:48:37.339781+0000","last_fullsized":"2024-09-12T02:48:37.339781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588143+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:37.588143+0000","last_peered":"2024-09-12T02:48:37.588143+0000","last_clean":"2024-09-12T02:48:37.588143+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:37.588143+0000","last_undegraded":"2024-09-12T02:48:37.588143+0000","last_fullsized":"2024-09-12T02:48:37.588143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339449+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:48:37.339449+0000","last_peered":"2024-09-12T02:48:37.339449+0000","last_clean":"2024-09-12T02:48:37.339449+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:48:37.339449+0000","last_undegraded":"2024-09-12T02:48:37.339449+0000","last_fullsized":"2024-09-12T02:48:37.339449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336705+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:48:37.336705+0000","last_peered":"2024-09-12T02:48:37.336705+0000","last_clean":"2024-09-12T02:48:37.336705+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:48:37.336705+0000","last_undegraded":"2024-09-12T02:48:37.336705+0000","last_fullsized":"2024-09-12T02:48:37.336705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341250+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:37.341250+0000","last_peered":"2024-09-12T02:48:37.341250+0000","last_clean":"2024-09-12T02:48:37.341250+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:37.341250+0000","last_undegraded":"2024-09-12T02:48:37.341250+0000","last_fullsized":"2024-09-12T02:48:37.341250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:48:37.341552+0000","last_peered":"2024-09-12T02:48:37.341552+0000","last_clean":"2024-09-12T02:48:37.341552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:48:37.341552+0000","last_undegraded":"2024-09-12T02:48:37.341552+0000","last_fullsized":"2024-09-12T02:48:37.341552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343136+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:48:37.343136+0000","last_peered":"2024-09-12T02:48:37.343136+0000","last_clean":"2024-09-12T02:48:37.343136+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:48:37.343136+0000","last_undegraded":"2024-09-12T02:48:37.343136+0000","last_fullsized":"2024-09-12T02:48:37.343136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587950+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:37.587950+0000","last_peered":"2024-09-12T02:48:37.587950+0000","last_clean":"2024-09-12T02:48:37.587950+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:37.587950+0000","last_undegraded":"2024-09-12T02:48:37.587950+0000","last_fullsized":"2024-09-12T02:48:37.587950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339541+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:48:37.339541+0000","last_peered":"2024-09-12T02:48:37.339541+0000","last_clean":"2024-09-12T02:48:37.339541+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:48:37.339541+0000","last_undegraded":"2024-09-12T02:48:37.339541+0000","last_fullsized":"2024-09-12T02:48:37.339541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588702+0000","last_change":"2024-09-12T02:48:37.588702+0000","last_active":"2024-09-12T02:48:37.588702+0000","last_peered":"2024-09-12T02:48:37.588702+0000","last_clean":"2024-09-12T02:48:37.588702+0000","last_became_active":"2024-09-12T02:48:37.588347+0000","last_became_peered":"2024-09-12T02:48:37.588347+0000","last_unstale":"2024-09-12T02:48:37.588702+0000","last_undegraded":"2024-09-12T02:48:37.588702+0000","last_fullsized":"2024-09-12T02:48:37.588702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:30.609070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586990+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:48:37.586990+0000","last_peered":"2024-09-12T02:48:37.586990+0000","last_clean":"2024-09-12T02:48:37.586990+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:48:37.586990+0000","last_undegraded":"2024-09-12T02:48:37.586990+0000","last_fullsized":"2024-09-12T02:48:37.586990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336464+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:48:37.336464+0000","last_peered":"2024-09-12T02:48:37.336464+0000","last_clean":"2024-09-12T02:48:37.336464+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:48:37.336464+0000","last_undegraded":"2024-09-12T02:48:37.336464+0000","last_fullsized":"2024-09-12T02:48:37.336464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337504+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:37.337504+0000","last_peered":"2024-09-12T02:48:37.337504+0000","last_clean":"2024-09-12T02:48:37.337504+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:37.337504+0000","last_undegraded":"2024-09-12T02:48:37.337504+0000","last_fullsized":"2024-09-12T02:48:37.337504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:38.132501+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:48:38.132501+0000","last_peered":"2024-09-12T02:48:38.132501+0000","last_clean":"2024-09-12T02:48:38.132501+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:48:38.132501+0000","last_undegraded":"2024-09-12T02:48:38.132501+0000","last_fullsized":"2024-09-12T02:48:38.132501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336272+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:48:37.336272+0000","last_peered":"2024-09-12T02:48:37.336272+0000","last_clean":"2024-09-12T02:48:37.336272+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:48:37.336272+0000","last_undegraded":"2024-09-12T02:48:37.336272+0000","last_fullsized":"2024-09-12T02:48:37.336272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588802+0000","last_change":"2024-09-12T02:48:37.588802+0000","last_active":"2024-09-12T02:48:37.588802+0000","last_peered":"2024-09-12T02:48:37.588802+0000","last_clean":"2024-09-12T02:48:37.588802+0000","last_became_active":"2024-09-12T02:48:37.588326+0000","last_became_peered":"2024-09-12T02:48:37.588326+0000","last_unstale":"2024-09-12T02:48:37.588802+0000","last_undegraded":"2024-09-12T02:48:37.588802+0000","last_fullsized":"2024-09-12T02:48:37.588802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586758+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:37.586758+0000","last_peered":"2024-09-12T02:48:37.586758+0000","last_clean":"2024-09-12T02:48:37.586758+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:37.586758+0000","last_undegraded":"2024-09-12T02:48:37.586758+0000","last_fullsized":"2024-09-12T02:48:37.586758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345128+0000","last_change":"2024-09-12T02:48:37.345128+0000","last_active":"2024-09-12T02:48:37.345128+0000","last_peered":"2024-09-12T02:48:37.345128+0000","last_clean":"2024-09-12T02:48:37.345128+0000","last_became_active":"2024-09-12T02:48:37.344670+0000","last_became_peered":"2024-09-12T02:48:37.344670+0000","last_unstale":"2024-09-12T02:48:37.345128+0000","last_undegraded":"2024-09-12T02:48:37.345128+0000","last_fullsized":"2024-09-12T02:48:37.345128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:32.206800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339329+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:48:37.339329+0000","last_peered":"2024-09-12T02:48:37.339329+0000","last_clean":"2024-09-12T02:48:37.339329+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:48:37.339329+0000","last_undegraded":"2024-09-12T02:48:37.339329+0000","last_fullsized":"2024-09-12T02:48:37.339329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341367+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:37.341367+0000","last_peered":"2024-09-12T02:48:37.341367+0000","last_clean":"2024-09-12T02:48:37.341367+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:37.341367+0000","last_undegraded":"2024-09-12T02:48:37.341367+0000","last_fullsized":"2024-09-12T02:48:37.341367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586906+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:48:37.586906+0000","last_peered":"2024-09-12T02:48:37.586906+0000","last_clean":"2024-09-12T02:48:37.586906+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:48:37.586906+0000","last_undegraded":"2024-09-12T02:48:37.586906+0000","last_fullsized":"2024-09-12T02:48:37.586906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586973+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:48:37.586973+0000","last_peered":"2024-09-12T02:48:37.586973+0000","last_clean":"2024-09-12T02:48:37.586973+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:48:37.586973+0000","last_undegraded":"2024-09-12T02:48:37.586973+0000","last_fullsized":"2024-09-12T02:48:37.586973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587180+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:48:37.587180+0000","last_peered":"2024-09-12T02:48:37.587180+0000","last_clean":"2024-09-12T02:48:37.587180+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:48:37.587180+0000","last_undegraded":"2024-09-12T02:48:37.587180+0000","last_fullsized":"2024-09-12T02:48:37.587180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343649+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:37.343649+0000","last_peered":"2024-09-12T02:48:37.343649+0000","last_clean":"2024-09-12T02:48:37.343649+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:37.343649+0000","last_undegraded":"2024-09-12T02:48:37.343649+0000","last_fullsized":"2024-09-12T02:48:37.343649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339501+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:48:37.339501+0000","last_peered":"2024-09-12T02:48:37.339501+0000","last_clean":"2024-09-12T02:48:37.339501+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:48:37.339501+0000","last_undegraded":"2024-09-12T02:48:37.339501+0000","last_fullsized":"2024-09-12T02:48:37.339501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587059+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:48:37.587059+0000","last_peered":"2024-09-12T02:48:37.587059+0000","last_clean":"2024-09-12T02:48:37.587059+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:48:37.587059+0000","last_undegraded":"2024-09-12T02:48:37.587059+0000","last_fullsized":"2024-09-12T02:48:37.587059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338318+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:37.338318+0000","last_peered":"2024-09-12T02:48:37.338318+0000","last_clean":"2024-09-12T02:48:37.338318+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:37.338318+0000","last_undegraded":"2024-09-12T02:48:37.338318+0000","last_fullsized":"2024-09-12T02:48:37.338318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.346098+0000","last_change":"2024-09-12T02:48:37.346098+0000","last_active":"2024-09-12T02:48:37.346098+0000","last_peered":"2024-09-12T02:48:37.346098+0000","last_clean":"2024-09-12T02:48:37.346098+0000","last_became_active":"2024-09-12T02:48:37.345696+0000","last_became_peered":"2024-09-12T02:48:37.345696+0000","last_unstale":"2024-09-12T02:48:37.346098+0000","last_undegraded":"2024-09-12T02:48:37.346098+0000","last_fullsized":"2024-09-12T02:48:37.346098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:48.040441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587347+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:48:37.587347+0000","last_peered":"2024-09-12T02:48:37.587347+0000","last_clean":"2024-09-12T02:48:37.587347+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:48:37.587347+0000","last_undegraded":"2024-09-12T02:48:37.587347+0000","last_fullsized":"2024-09-12T02:48:37.587347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336481+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:48:37.336481+0000","last_peered":"2024-09-12T02:48:37.336481+0000","last_clean":"2024-09-12T02:48:37.336481+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:48:37.336481+0000","last_undegraded":"2024-09-12T02:48:37.336481+0000","last_fullsized":"2024-09-12T02:48:37.336481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337296+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:37.337296+0000","last_peered":"2024-09-12T02:48:37.337296+0000","last_clean":"2024-09-12T02:48:37.337296+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:37.337296+0000","last_undegraded":"2024-09-12T02:48:37.337296+0000","last_fullsized":"2024-09-12T02:48:37.337296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341055+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:48:37.341055+0000","last_peered":"2024-09-12T02:48:37.341055+0000","last_clean":"2024-09-12T02:48:37.341055+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:48:37.341055+0000","last_undegraded":"2024-09-12T02:48:37.341055+0000","last_fullsized":"2024-09-12T02:48:37.341055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340312+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:37.340312+0000","last_peered":"2024-09-12T02:48:37.340312+0000","last_clean":"2024-09-12T02:48:37.340312+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:37.340312+0000","last_undegraded":"2024-09-12T02:48:37.340312+0000","last_fullsized":"2024-09-12T02:48:37.340312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337321+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:37.337321+0000","last_peered":"2024-09-12T02:48:37.337321+0000","last_clean":"2024-09-12T02:48:37.337321+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:37.337321+0000","last_undegraded":"2024-09-12T02:48:37.337321+0000","last_fullsized":"2024-09-12T02:48:37.337321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337404+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:37.337404+0000","last_peered":"2024-09-12T02:48:37.337404+0000","last_clean":"2024-09-12T02:48:37.337404+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:37.337404+0000","last_undegraded":"2024-09-12T02:48:37.337404+0000","last_fullsized":"2024-09-12T02:48:37.337404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339395+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:37.339395+0000","last_peered":"2024-09-12T02:48:37.339395+0000","last_clean":"2024-09-12T02:48:37.339395+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:37.339395+0000","last_undegraded":"2024-09-12T02:48:37.339395+0000","last_fullsized":"2024-09-12T02:48:37.339395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.344145+0000","last_change":"2024-09-12T02:48:37.344145+0000","last_active":"2024-09-12T02:48:37.344145+0000","last_peered":"2024-09-12T02:48:37.344145+0000","last_clean":"2024-09-12T02:48:37.344145+0000","last_became_active":"2024-09-12T02:48:37.343694+0000","last_became_peered":"2024-09-12T02:48:37.343694+0000","last_unstale":"2024-09-12T02:48:37.344145+0000","last_undegraded":"2024-09-12T02:48:37.344145+0000","last_fullsized":"2024-09-12T02:48:37.344145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:55.742228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586644+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:48:37.586644+0000","last_peered":"2024-09-12T02:48:37.586644+0000","last_clean":"2024-09-12T02:48:37.586644+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:48:37.586644+0000","last_undegraded":"2024-09-12T02:48:37.586644+0000","last_fullsized":"2024-09-12T02:48:37.586644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:40.135425+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:48:40.135425+0000","last_peered":"2024-09-12T02:48:40.135425+0000","last_clean":"2024-09-12T02:48:40.135425+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:48:40.135425+0000","last_undegraded":"2024-09-12T02:48:40.135425+0000","last_fullsized":"2024-09-12T02:48:40.135425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587962+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:37.587962+0000","last_peered":"2024-09-12T02:48:37.587962+0000","last_clean":"2024-09-12T02:48:37.587962+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:37.587962+0000","last_undegraded":"2024-09-12T02:48:37.587962+0000","last_fullsized":"2024-09-12T02:48:37.587962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587055+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:48:37.587055+0000","last_peered":"2024-09-12T02:48:37.587055+0000","last_clean":"2024-09-12T02:48:37.587055+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:48:37.587055+0000","last_undegraded":"2024-09-12T02:48:37.587055+0000","last_fullsized":"2024-09-12T02:48:37.587055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337741+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:37.337741+0000","last_peered":"2024-09-12T02:48:37.337741+0000","last_clean":"2024-09-12T02:48:37.337741+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:37.337741+0000","last_undegraded":"2024-09-12T02:48:37.337741+0000","last_fullsized":"2024-09-12T02:48:37.337741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341156+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:48:37.341156+0000","last_peered":"2024-09-12T02:48:37.341156+0000","last_clean":"2024-09-12T02:48:37.341156+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:48:37.341156+0000","last_undegraded":"2024-09-12T02:48:37.341156+0000","last_fullsized":"2024-09-12T02:48:37.341156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336133+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:48:37.336133+0000","last_peered":"2024-09-12T02:48:37.336133+0000","last_clean":"2024-09-12T02:48:37.336133+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:48:37.336133+0000","last_undegraded":"2024-09-12T02:48:37.336133+0000","last_fullsized":"2024-09-12T02:48:37.336133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.353056+0000","last_change":"2024-09-12T02:48:37.352674+0000","last_active":"2024-09-12T02:48:37.353056+0000","last_peered":"2024-09-12T02:48:37.353056+0000","last_clean":"2024-09-12T02:48:37.353056+0000","last_became_active":"2024-09-12T02:48:37.345293+0000","last_became_peered":"2024-09-12T02:48:37.345293+0000","last_unstale":"2024-09-12T02:48:37.353056+0000","last_undegraded":"2024-09-12T02:48:37.353056+0000","last_fullsized":"2024-09-12T02:48:37.353056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:00.609598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338121+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:37.338121+0000","last_peered":"2024-09-12T02:48:37.338121+0000","last_clean":"2024-09-12T02:48:37.338121+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:37.338121+0000","last_undegraded":"2024-09-12T02:48:37.338121+0000","last_fullsized":"2024-09-12T02:48:37.338121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340161+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:48:37.340161+0000","last_peered":"2024-09-12T02:48:37.340161+0000","last_clean":"2024-09-12T02:48:37.340161+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:48:37.340161+0000","last_undegraded":"2024-09-12T02:48:37.340161+0000","last_fullsized":"2024-09-12T02:48:37.340161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'31","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:41.139823+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:48:41.139823+0000","last_peered":"2024-09-12T02:48:41.139823+0000","last_clean":"2024-09-12T02:48:41.139823+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:48:41.139823+0000","last_undegraded":"2024-09-12T02:48:41.139823+0000","last_fullsized":"2024-09-12T02:48:41.139823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58661,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":52,"num_read_kb":83,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587651+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:37.587651+0000","last_peered":"2024-09-12T02:48:37.587651+0000","last_clean":"2024-09-12T02:48:37.587651+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:37.587651+0000","last_undegraded":"2024-09-12T02:48:37.587651+0000","last_fullsized":"2024-09-12T02:48:37.587651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339610+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:48:37.339610+0000","last_peered":"2024-09-12T02:48:37.339610+0000","last_clean":"2024-09-12T02:48:37.339610+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:48:37.339610+0000","last_undegraded":"2024-09-12T02:48:37.339610+0000","last_fullsized":"2024-09-12T02:48:37.339610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338294+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:37.338294+0000","last_peered":"2024-09-12T02:48:37.338294+0000","last_clean":"2024-09-12T02:48:37.338294+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:37.338294+0000","last_undegraded":"2024-09-12T02:48:37.338294+0000","last_fullsized":"2024-09-12T02:48:37.338294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341022+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:37.341022+0000","last_peered":"2024-09-12T02:48:37.341022+0000","last_clean":"2024-09-12T02:48:37.341022+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:37.341022+0000","last_undegraded":"2024-09-12T02:48:37.341022+0000","last_fullsized":"2024-09-12T02:48:37.341022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341169+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:48:37.341169+0000","last_peered":"2024-09-12T02:48:37.341169+0000","last_clean":"2024-09-12T02:48:37.341169+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:48:37.341169+0000","last_undegraded":"2024-09-12T02:48:37.341169+0000","last_fullsized":"2024-09-12T02:48:37.341169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587207+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:37.587207+0000","last_peered":"2024-09-12T02:48:37.587207+0000","last_clean":"2024-09-12T02:48:37.587207+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:37.587207+0000","last_undegraded":"2024-09-12T02:48:37.587207+0000","last_fullsized":"2024-09-12T02:48:37.587207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587311+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:37.587311+0000","last_peered":"2024-09-12T02:48:37.587311+0000","last_clean":"2024-09-12T02:48:37.587311+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:37.587311+0000","last_undegraded":"2024-09-12T02:48:37.587311+0000","last_fullsized":"2024-09-12T02:48:37.587311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340162+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:48:37.340162+0000","last_peered":"2024-09-12T02:48:37.340162+0000","last_clean":"2024-09-12T02:48:37.340162+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:48:37.340162+0000","last_undegraded":"2024-09-12T02:48:37.340162+0000","last_fullsized":"2024-09-12T02:48:37.340162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:40.134304+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:48:40.134304+0000","last_peered":"2024-09-12T02:48:40.134304+0000","last_clean":"2024-09-12T02:48:40.134304+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:48:40.134304+0000","last_undegraded":"2024-09-12T02:48:40.134304+0000","last_fullsized":"2024-09-12T02:48:40.134304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339163+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:48:37.339163+0000","last_peered":"2024-09-12T02:48:37.339163+0000","last_clean":"2024-09-12T02:48:37.339163+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:48:37.339163+0000","last_undegraded":"2024-09-12T02:48:37.339163+0000","last_fullsized":"2024-09-12T02:48:37.339163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340919+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:48:37.340919+0000","last_peered":"2024-09-12T02:48:37.340919+0000","last_clean":"2024-09-12T02:48:37.340919+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:48:37.340919+0000","last_undegraded":"2024-09-12T02:48:37.340919+0000","last_fullsized":"2024-09-12T02:48:37.340919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587478+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:37.587478+0000","last_peered":"2024-09-12T02:48:37.587478+0000","last_clean":"2024-09-12T02:48:37.587478+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:37.587478+0000","last_undegraded":"2024-09-12T02:48:37.587478+0000","last_fullsized":"2024-09-12T02:48:37.587478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337681+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:37.337681+0000","last_peered":"2024-09-12T02:48:37.337681+0000","last_clean":"2024-09-12T02:48:37.337681+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:37.337681+0000","last_undegraded":"2024-09-12T02:48:37.337681+0000","last_fullsized":"2024-09-12T02:48:37.337681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340797+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:48:37.340797+0000","last_peered":"2024-09-12T02:48:37.340797+0000","last_clean":"2024-09-12T02:48:37.340797+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:48:37.340797+0000","last_undegraded":"2024-09-12T02:48:37.340797+0000","last_fullsized":"2024-09-12T02:48:37.340797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339975+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:48:37.339975+0000","last_peered":"2024-09-12T02:48:37.339975+0000","last_clean":"2024-09-12T02:48:37.339975+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:48:37.339975+0000","last_undegraded":"2024-09-12T02:48:37.339975+0000","last_fullsized":"2024-09-12T02:48:37.339975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336024+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:37.336024+0000","last_peered":"2024-09-12T02:48:37.336024+0000","last_clean":"2024-09-12T02:48:37.336024+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:37.336024+0000","last_undegraded":"2024-09-12T02:48:37.336024+0000","last_fullsized":"2024-09-12T02:48:37.336024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343029+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:37.343029+0000","last_peered":"2024-09-12T02:48:37.343029+0000","last_clean":"2024-09-12T02:48:37.343029+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:37.343029+0000","last_undegraded":"2024-09-12T02:48:37.343029+0000","last_fullsized":"2024-09-12T02:48:37.343029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337742+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:37.337742+0000","last_peered":"2024-09-12T02:48:37.337742+0000","last_clean":"2024-09-12T02:48:37.337742+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:37.337742+0000","last_undegraded":"2024-09-12T02:48:37.337742+0000","last_fullsized":"2024-09-12T02:48:37.337742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343312+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:48:37.343312+0000","last_peered":"2024-09-12T02:48:37.343312+0000","last_clean":"2024-09-12T02:48:37.343312+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:48:37.343312+0000","last_undegraded":"2024-09-12T02:48:37.343312+0000","last_fullsized":"2024-09-12T02:48:37.343312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:41.135934+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:48:41.135934+0000","last_peered":"2024-09-12T02:48:41.135934+0000","last_clean":"2024-09-12T02:48:41.135934+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:48:41.135934+0000","last_undegraded":"2024-09-12T02:48:41.135934+0000","last_fullsized":"2024-09-12T02:48:41.135934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588956+0000","last_change":"2024-09-12T02:48:37.588956+0000","last_active":"2024-09-12T02:48:37.588956+0000","last_peered":"2024-09-12T02:48:37.588956+0000","last_clean":"2024-09-12T02:48:37.588956+0000","last_became_active":"2024-09-12T02:48:37.588561+0000","last_became_peered":"2024-09-12T02:48:37.588561+0000","last_unstale":"2024-09-12T02:48:37.588956+0000","last_undegraded":"2024-09-12T02:48:37.588956+0000","last_fullsized":"2024-09-12T02:48:37.588956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:08.645076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343699+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:48:37.343699+0000","last_peered":"2024-09-12T02:48:37.343699+0000","last_clean":"2024-09-12T02:48:37.343699+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:48:37.343699+0000","last_undegraded":"2024-09-12T02:48:37.343699+0000","last_fullsized":"2024-09-12T02:48:37.343699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340641+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:48:37.340641+0000","last_peered":"2024-09-12T02:48:37.340641+0000","last_clean":"2024-09-12T02:48:37.340641+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:48:37.340641+0000","last_undegraded":"2024-09-12T02:48:37.340641+0000","last_fullsized":"2024-09-12T02:48:37.340641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343329+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:48:37.343329+0000","last_peered":"2024-09-12T02:48:37.343329+0000","last_clean":"2024-09-12T02:48:37.343329+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:48:37.343329+0000","last_undegraded":"2024-09-12T02:48:37.343329+0000","last_fullsized":"2024-09-12T02:48:37.343329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.589056+0000","last_change":"2024-09-12T02:48:37.589056+0000","last_active":"2024-09-12T02:48:37.589056+0000","last_peered":"2024-09-12T02:48:37.589056+0000","last_clean":"2024-09-12T02:48:37.589056+0000","last_became_active":"2024-09-12T02:48:37.588629+0000","last_became_peered":"2024-09-12T02:48:37.588629+0000","last_unstale":"2024-09-12T02:48:37.589056+0000","last_undegraded":"2024-09-12T02:48:37.589056+0000","last_fullsized":"2024-09-12T02:48:37.589056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:03.490341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340249+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:48:37.340249+0000","last_peered":"2024-09-12T02:48:37.340249+0000","last_clean":"2024-09-12T02:48:37.340249+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:48:37.340249+0000","last_undegraded":"2024-09-12T02:48:37.340249+0000","last_fullsized":"2024-09-12T02:48:37.340249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336590+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:48:37.336590+0000","last_peered":"2024-09-12T02:48:37.336590+0000","last_clean":"2024-09-12T02:48:37.336590+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:48:37.336590+0000","last_undegraded":"2024-09-12T02:48:37.336590+0000","last_fullsized":"2024-09-12T02:48:37.336590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341713+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:48:37.341713+0000","last_peered":"2024-09-12T02:48:37.341713+0000","last_clean":"2024-09-12T02:48:37.341713+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:48:37.341713+0000","last_undegraded":"2024-09-12T02:48:37.341713+0000","last_fullsized":"2024-09-12T02:48:37.341713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343987+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:48:37.343987+0000","last_peered":"2024-09-12T02:48:37.343987+0000","last_clean":"2024-09-12T02:48:37.343987+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:48:37.343987+0000","last_undegraded":"2024-09-12T02:48:37.343987+0000","last_fullsized":"2024-09-12T02:48:37.343987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345556+0000","last_change":"2024-09-12T02:48:37.345556+0000","last_active":"2024-09-12T02:48:37.345556+0000","last_peered":"2024-09-12T02:48:37.345556+0000","last_clean":"2024-09-12T02:48:37.345556+0000","last_became_active":"2024-09-12T02:48:37.345220+0000","last_became_peered":"2024-09-12T02:48:37.345220+0000","last_unstale":"2024-09-12T02:48:37.345556+0000","last_undegraded":"2024-09-12T02:48:37.345556+0000","last_fullsized":"2024-09-12T02:48:37.345556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:25.562372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337339+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:37.337339+0000","last_peered":"2024-09-12T02:48:37.337339+0000","last_clean":"2024-09-12T02:48:37.337339+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:37.337339+0000","last_undegraded":"2024-09-12T02:48:37.337339+0000","last_fullsized":"2024-09-12T02:48:37.337339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339282+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:48:37.339282+0000","last_peered":"2024-09-12T02:48:37.339282+0000","last_clean":"2024-09-12T02:48:37.339282+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:48:37.339282+0000","last_undegraded":"2024-09-12T02:48:37.339282+0000","last_fullsized":"2024-09-12T02:48:37.339282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339814+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:48:37.339814+0000","last_peered":"2024-09-12T02:48:37.339814+0000","last_clean":"2024-09-12T02:48:37.339814+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:48:37.339814+0000","last_undegraded":"2024-09-12T02:48:37.339814+0000","last_fullsized":"2024-09-12T02:48:37.339814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336265+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:48:37.336265+0000","last_peered":"2024-09-12T02:48:37.336265+0000","last_clean":"2024-09-12T02:48:37.336265+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:48:37.336265+0000","last_undegraded":"2024-09-12T02:48:37.336265+0000","last_fullsized":"2024-09-12T02:48:37.336265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341263+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:48:37.341263+0000","last_peered":"2024-09-12T02:48:37.341263+0000","last_clean":"2024-09-12T02:48:37.341263+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:48:37.341263+0000","last_undegraded":"2024-09-12T02:48:37.341263+0000","last_fullsized":"2024-09-12T02:48:37.341263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343830+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:48:37.343830+0000","last_peered":"2024-09-12T02:48:37.343830+0000","last_clean":"2024-09-12T02:48:37.343830+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:48:37.343830+0000","last_undegraded":"2024-09-12T02:48:37.343830+0000","last_fullsized":"2024-09-12T02:48:37.343830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588060+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:37.588060+0000","last_peered":"2024-09-12T02:48:37.588060+0000","last_clean":"2024-09-12T02:48:37.588060+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:37.588060+0000","last_undegraded":"2024-09-12T02:48:37.588060+0000","last_fullsized":"2024-09-12T02:48:37.588060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342161+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:48:37.342161+0000","last_peered":"2024-09-12T02:48:37.342161+0000","last_clean":"2024-09-12T02:48:37.342161+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:48:37.342161+0000","last_undegraded":"2024-09-12T02:48:37.342161+0000","last_fullsized":"2024-09-12T02:48:37.342161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339233+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:48:37.339233+0000","last_peered":"2024-09-12T02:48:37.339233+0000","last_clean":"2024-09-12T02:48:37.339233+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:48:37.339233+0000","last_undegraded":"2024-09-12T02:48:37.339233+0000","last_fullsized":"2024-09-12T02:48:37.339233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339694+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:48:37.339694+0000","last_peered":"2024-09-12T02:48:37.339694+0000","last_clean":"2024-09-12T02:48:37.339694+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:48:37.339694+0000","last_undegraded":"2024-09-12T02:48:37.339694+0000","last_fullsized":"2024-09-12T02:48:37.339694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339992+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:48:37.339992+0000","last_peered":"2024-09-12T02:48:37.339992+0000","last_clean":"2024-09-12T02:48:37.339992+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:48:37.339992+0000","last_undegraded":"2024-09-12T02:48:37.339992+0000","last_fullsized":"2024-09-12T02:48:37.339992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336229+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:48:37.336229+0000","last_peered":"2024-09-12T02:48:37.336229+0000","last_clean":"2024-09-12T02:48:37.336229+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:48:37.336229+0000","last_undegraded":"2024-09-12T02:48:37.336229+0000","last_fullsized":"2024-09-12T02:48:37.336229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341015+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:37.341015+0000","last_peered":"2024-09-12T02:48:37.341015+0000","last_clean":"2024-09-12T02:48:37.341015+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:37.341015+0000","last_undegraded":"2024-09-12T02:48:37.341015+0000","last_fullsized":"2024-09-12T02:48:37.341015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587114+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:48:37.587114+0000","last_peered":"2024-09-12T02:48:37.587114+0000","last_clean":"2024-09-12T02:48:37.587114+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:48:37.587114+0000","last_undegraded":"2024-09-12T02:48:37.587114+0000","last_fullsized":"2024-09-12T02:48:37.587114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343668+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:48:37.343668+0000","last_peered":"2024-09-12T02:48:37.343668+0000","last_clean":"2024-09-12T02:48:37.343668+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:48:37.343668+0000","last_undegraded":"2024-09-12T02:48:37.343668+0000","last_fullsized":"2024-09-12T02:48:37.343668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340009+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:48:37.340009+0000","last_peered":"2024-09-12T02:48:37.340009+0000","last_clean":"2024-09-12T02:48:37.340009+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:48:37.340009+0000","last_undegraded":"2024-09-12T02:48:37.340009+0000","last_fullsized":"2024-09-12T02:48:37.340009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339344+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:48:37.339344+0000","last_peered":"2024-09-12T02:48:37.339344+0000","last_clean":"2024-09-12T02:48:37.339344+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:48:37.339344+0000","last_undegraded":"2024-09-12T02:48:37.339344+0000","last_fullsized":"2024-09-12T02:48:37.339344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586757+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:48:37.586757+0000","last_peered":"2024-09-12T02:48:37.586757+0000","last_clean":"2024-09-12T02:48:37.586757+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:48:37.586757+0000","last_undegraded":"2024-09-12T02:48:37.586757+0000","last_fullsized":"2024-09-12T02:48:37.586757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342802+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:48:37.342802+0000","last_peered":"2024-09-12T02:48:37.342802+0000","last_clean":"2024-09-12T02:48:37.342802+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:48:37.342802+0000","last_undegraded":"2024-09-12T02:48:37.342802+0000","last_fullsized":"2024-09-12T02:48:37.342802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339808+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:48:37.339808+0000","last_peered":"2024-09-12T02:48:37.339808+0000","last_clean":"2024-09-12T02:48:37.339808+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:48:37.339808+0000","last_undegraded":"2024-09-12T02:48:37.339808+0000","last_fullsized":"2024-09-12T02:48:37.339808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340891+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:37.340891+0000","last_peered":"2024-09-12T02:48:37.340891+0000","last_clean":"2024-09-12T02:48:37.340891+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:37.340891+0000","last_undegraded":"2024-09-12T02:48:37.340891+0000","last_fullsized":"2024-09-12T02:48:37.340891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340897+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:48:37.340897+0000","last_peered":"2024-09-12T02:48:37.340897+0000","last_clean":"2024-09-12T02:48:37.340897+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:48:37.340897+0000","last_undegraded":"2024-09-12T02:48:37.340897+0000","last_fullsized":"2024-09-12T02:48:37.340897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341040+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:48:37.341040+0000","last_peered":"2024-09-12T02:48:37.341040+0000","last_clean":"2024-09-12T02:48:37.341040+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:48:37.341040+0000","last_undegraded":"2024-09-12T02:48:37.341040+0000","last_fullsized":"2024-09-12T02:48:37.341040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342054+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:48:37.342054+0000","last_peered":"2024-09-12T02:48:37.342054+0000","last_clean":"2024-09-12T02:48:37.342054+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:48:37.342054+0000","last_undegraded":"2024-09-12T02:48:37.342054+0000","last_fullsized":"2024-09-12T02:48:37.342054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":158,"num_read_kb":144,"num_write":167,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":501459,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1126400,"data_stored":1147354,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":505757,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":501581,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1088128,"data_compressed":56827,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":496837,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1056768,"data_stored":1093080,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":501487,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":497031,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":557117,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":555408,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":488861,"data_compressed":20581,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:48:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:44 smithi145 ceph-mon[34674]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 9 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:45.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:45 smithi196 ceph-mon[34587]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:45.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:45 smithi046 ceph-mon[30951]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:45.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:45 smithi145 ceph-mon[34674]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:46 smithi196 ceph-mon[34587]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:46.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:46 smithi046 ceph-mon[30951]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:46 smithi145 ceph-mon[34674]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:47.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:48:47.838 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 -- 172.21.15.46:0/629825403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c101400 msgr2=0x7f188c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 --2- 172.21.15.46:0/629825403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c101400 0x7f188c10d930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1884009cd0 tx=0x7f188402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 -- 172.21.15.46:0/629825403 shutdown_connections 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 --2- 172.21.15.46:0/629825403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c101400 0x7f188c10d930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 --2- 172.21.15.46:0/629825403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c100a60 0x7f188c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 --2- 172.21.15.46:0/629825403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c106a60 0x7f188c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 -- 172.21.15.46:0/629825403 >> 172.21.15.46:0/629825403 conn(0x7f188c0fc780 msgr2=0x7f188c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 -- 172.21.15.46:0/629825403 shutdown_connections 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.367+0000 7f1890c26640 1 -- 172.21.15.46:0/629825403 wait complete. 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 Processor -- start 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 -- start start 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c101400 0x7f188c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c106a60 0x7f188c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f188c110d10 con 0x7f188c100a60 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f188c110b90 con 0x7f188c101400 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f1890c26640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f188c110e90 con 0x7f188c106a60 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34748/0 (socket says 172.21.15.46:34748) 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 -- 172.21.15.46:0/3731386969 learned_addr learned my addr 172.21.15.46:0/3731386969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188ad9d640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c106a60 0x7f188c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 -- 172.21.15.46:0/3731386969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c106a60 msgr2=0x7f188c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c106a60 0x7f188c1a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.368+0000 7f188a59c640 1 -- 172.21.15.46:0/3731386969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c101400 msgr2=0x7f188c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f1889d9b640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c101400 0x7f188c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f188a59c640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c101400 0x7f188c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f188a59c640 1 -- 172.21.15.46:0/3731386969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1a47c0 con 0x7f188c100a60 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f188a59c640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d950 tx=0x7f187c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c017030 con 0x7f188c100a60 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c1a4990 con 0x7f188c100a60 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f187c004020 con 0x7f188c100a60 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c0047a0 con 0x7f188c100a60 2024-09-12T02:48:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.369+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1a4ed0 con 0x7f188c100a60 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.370+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c100ec0 con 0x7f188c100a60 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.371+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f187c0041d0 con 0x7f188c100a60 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.372+0000 7f187b7fe640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 0x7f18740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.372+0000 7f1889d9b640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 0x7f18740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.372+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f187c0cefc0 con 0x7f188c100a60 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.372+0000 7f1889d9b640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 0x7f18740a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1880005fd0 tx=0x7f1880005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.376+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f187c09ba00 con 0x7f188c100a60 2024-09-12T02:48:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:48 smithi196 ceph-mon[34587]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 618 B/s wr, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:48.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:48 smithi046 ceph-mon[30951]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 618 B/s wr, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.548+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f188c062630 con 0x7f18740a2c90 2024-09-12T02:48:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.557+0000 7f187b7fe640 1 -- 172.21.15.46:0/3731386969 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340586 (secure 0 0 0) 0x7f188c062630 con 0x7f18740a2c90 2024-09-12T02:48:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:48.562 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 msgr2=0x7f18740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 0x7f18740a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1880005fd0 tx=0x7f1880005850 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 msgr2=0x7f188c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d950 tx=0x7f187c00de10 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 shutdown_connections 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f18740a2c90 0x7f18740a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f188c106a60 0x7f188c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f188c101400 0x7f188c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 --2- 172.21.15.46:0/3731386969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f188c100a60 0x7f188c199200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.559+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 >> 172.21.15.46:0/3731386969 conn(0x7f188c0fc780 msgr2=0x7f188c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.560+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 shutdown_connections 2024-09-12T02:48:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:48.560+0000 7f1890c26640 1 -- 172.21.15.46:0/3731386969 wait complete. 2024-09-12T02:48:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:48 smithi145 ceph-mon[34674]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 618 B/s wr, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T02:48:49.377 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-12T02:48:48.276304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":325,"num_read_kb":284,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369644,"kb_used_data":12020,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572180,"statfs":{"total":1151940427776,"available":1151561912320,"internally_reserved":0,"allocated":12308480,"data_stored":7969037,"data_compressed":391518,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":41,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.941377"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588213+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:37.588213+0000","last_peered":"2024-09-12T02:48:37.588213+0000","last_clean":"2024-09-12T02:48:37.588213+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:37.588213+0000","last_undegraded":"2024-09-12T02:48:37.588213+0000","last_fullsized":"2024-09-12T02:48:37.588213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339671+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:48:37.339671+0000","last_peered":"2024-09-12T02:48:37.339671+0000","last_clean":"2024-09-12T02:48:37.339671+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:48:37.339671+0000","last_undegraded":"2024-09-12T02:48:37.339671+0000","last_fullsized":"2024-09-12T02:48:37.339671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343048+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:48:37.343048+0000","last_peered":"2024-09-12T02:48:37.343048+0000","last_clean":"2024-09-12T02:48:37.343048+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:48:37.343048+0000","last_undegraded":"2024-09-12T02:48:37.343048+0000","last_fullsized":"2024-09-12T02:48:37.343048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339882+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:48:37.339882+0000","last_peered":"2024-09-12T02:48:37.339882+0000","last_clean":"2024-09-12T02:48:37.339882+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:48:37.339882+0000","last_undegraded":"2024-09-12T02:48:37.339882+0000","last_fullsized":"2024-09-12T02:48:37.339882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341105+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:48:37.341105+0000","last_peered":"2024-09-12T02:48:37.341105+0000","last_clean":"2024-09-12T02:48:37.341105+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:48:37.341105+0000","last_undegraded":"2024-09-12T02:48:37.341105+0000","last_fullsized":"2024-09-12T02:48:37.341105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340351+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:37.340351+0000","last_peered":"2024-09-12T02:48:37.340351+0000","last_clean":"2024-09-12T02:48:37.340351+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:37.340351+0000","last_undegraded":"2024-09-12T02:48:37.340351+0000","last_fullsized":"2024-09-12T02:48:37.340351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341138+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:37.341138+0000","last_peered":"2024-09-12T02:48:37.341138+0000","last_clean":"2024-09-12T02:48:37.341138+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:37.341138+0000","last_undegraded":"2024-09-12T02:48:37.341138+0000","last_fullsized":"2024-09-12T02:48:37.341138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339509+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:48:37.339509+0000","last_peered":"2024-09-12T02:48:37.339509+0000","last_clean":"2024-09-12T02:48:37.339509+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:48:37.339509+0000","last_undegraded":"2024-09-12T02:48:37.339509+0000","last_fullsized":"2024-09-12T02:48:37.339509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341184+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:37.341184+0000","last_peered":"2024-09-12T02:48:37.341184+0000","last_clean":"2024-09-12T02:48:37.341184+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:37.341184+0000","last_undegraded":"2024-09-12T02:48:37.341184+0000","last_fullsized":"2024-09-12T02:48:37.341184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340619+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:48:37.340619+0000","last_peered":"2024-09-12T02:48:37.340619+0000","last_clean":"2024-09-12T02:48:37.340619+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:48:37.340619+0000","last_undegraded":"2024-09-12T02:48:37.340619+0000","last_fullsized":"2024-09-12T02:48:37.340619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338203+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:37.338203+0000","last_peered":"2024-09-12T02:48:37.338203+0000","last_clean":"2024-09-12T02:48:37.338203+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:37.338203+0000","last_undegraded":"2024-09-12T02:48:37.338203+0000","last_fullsized":"2024-09-12T02:48:37.338203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340502+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:37.340502+0000","last_peered":"2024-09-12T02:48:37.340502+0000","last_clean":"2024-09-12T02:48:37.340502+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:37.340502+0000","last_undegraded":"2024-09-12T02:48:37.340502+0000","last_fullsized":"2024-09-12T02:48:37.340502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586858+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:48:37.586858+0000","last_peered":"2024-09-12T02:48:37.586858+0000","last_clean":"2024-09-12T02:48:37.586858+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:48:37.586858+0000","last_undegraded":"2024-09-12T02:48:37.586858+0000","last_fullsized":"2024-09-12T02:48:37.586858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339937+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:37.339937+0000","last_peered":"2024-09-12T02:48:37.339937+0000","last_clean":"2024-09-12T02:48:37.339937+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:37.339937+0000","last_undegraded":"2024-09-12T02:48:37.339937+0000","last_fullsized":"2024-09-12T02:48:37.339937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340671+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:37.340671+0000","last_peered":"2024-09-12T02:48:37.340671+0000","last_clean":"2024-09-12T02:48:37.340671+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:37.340671+0000","last_undegraded":"2024-09-12T02:48:37.340671+0000","last_fullsized":"2024-09-12T02:48:37.340671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339210+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:48:37.339210+0000","last_peered":"2024-09-12T02:48:37.339210+0000","last_clean":"2024-09-12T02:48:37.339210+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:48:37.339210+0000","last_undegraded":"2024-09-12T02:48:37.339210+0000","last_fullsized":"2024-09-12T02:48:37.339210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343983+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:48:37.343983+0000","last_peered":"2024-09-12T02:48:37.343983+0000","last_clean":"2024-09-12T02:48:37.343983+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:48:37.343983+0000","last_undegraded":"2024-09-12T02:48:37.343983+0000","last_fullsized":"2024-09-12T02:48:37.343983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341063+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:37.341063+0000","last_peered":"2024-09-12T02:48:37.341063+0000","last_clean":"2024-09-12T02:48:37.341063+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:37.341063+0000","last_undegraded":"2024-09-12T02:48:37.341063+0000","last_fullsized":"2024-09-12T02:48:37.341063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340770+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:48:37.340770+0000","last_peered":"2024-09-12T02:48:37.340770+0000","last_clean":"2024-09-12T02:48:37.340770+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:48:37.340770+0000","last_undegraded":"2024-09-12T02:48:37.340770+0000","last_fullsized":"2024-09-12T02:48:37.340770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339382+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:37.339382+0000","last_peered":"2024-09-12T02:48:37.339382+0000","last_clean":"2024-09-12T02:48:37.339382+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:37.339382+0000","last_undegraded":"2024-09-12T02:48:37.339382+0000","last_fullsized":"2024-09-12T02:48:37.339382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588035+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:37.588035+0000","last_peered":"2024-09-12T02:48:37.588035+0000","last_clean":"2024-09-12T02:48:37.588035+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:37.588035+0000","last_undegraded":"2024-09-12T02:48:37.588035+0000","last_fullsized":"2024-09-12T02:48:37.588035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339398+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:48:37.339398+0000","last_peered":"2024-09-12T02:48:37.339398+0000","last_clean":"2024-09-12T02:48:37.339398+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:48:37.339398+0000","last_undegraded":"2024-09-12T02:48:37.339398+0000","last_fullsized":"2024-09-12T02:48:37.339398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339366+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:37.339366+0000","last_peered":"2024-09-12T02:48:37.339366+0000","last_clean":"2024-09-12T02:48:37.339366+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:37.339366+0000","last_undegraded":"2024-09-12T02:48:37.339366+0000","last_fullsized":"2024-09-12T02:48:37.339366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.590486+0000","last_change":"2024-09-12T02:48:37.590486+0000","last_active":"2024-09-12T02:48:37.590486+0000","last_peered":"2024-09-12T02:48:37.590486+0000","last_clean":"2024-09-12T02:48:37.590486+0000","last_became_active":"2024-09-12T02:48:37.589930+0000","last_became_peered":"2024-09-12T02:48:37.589930+0000","last_unstale":"2024-09-12T02:48:37.590486+0000","last_undegraded":"2024-09-12T02:48:37.590486+0000","last_fullsized":"2024-09-12T02:48:37.590486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:42.323965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341047+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:48:37.341047+0000","last_peered":"2024-09-12T02:48:37.341047+0000","last_clean":"2024-09-12T02:48:37.341047+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:48:37.341047+0000","last_undegraded":"2024-09-12T02:48:37.341047+0000","last_fullsized":"2024-09-12T02:48:37.341047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340953+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:37.340953+0000","last_peered":"2024-09-12T02:48:37.340953+0000","last_clean":"2024-09-12T02:48:37.340953+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:37.340953+0000","last_undegraded":"2024-09-12T02:48:37.340953+0000","last_fullsized":"2024-09-12T02:48:37.340953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343184+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:48:37.343184+0000","last_peered":"2024-09-12T02:48:37.343184+0000","last_clean":"2024-09-12T02:48:37.343184+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:48:37.343184+0000","last_undegraded":"2024-09-12T02:48:37.343184+0000","last_fullsized":"2024-09-12T02:48:37.343184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339781+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:48:37.339781+0000","last_peered":"2024-09-12T02:48:37.339781+0000","last_clean":"2024-09-12T02:48:37.339781+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:48:37.339781+0000","last_undegraded":"2024-09-12T02:48:37.339781+0000","last_fullsized":"2024-09-12T02:48:37.339781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588143+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:37.588143+0000","last_peered":"2024-09-12T02:48:37.588143+0000","last_clean":"2024-09-12T02:48:37.588143+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:37.588143+0000","last_undegraded":"2024-09-12T02:48:37.588143+0000","last_fullsized":"2024-09-12T02:48:37.588143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339449+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:48:37.339449+0000","last_peered":"2024-09-12T02:48:37.339449+0000","last_clean":"2024-09-12T02:48:37.339449+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:48:37.339449+0000","last_undegraded":"2024-09-12T02:48:37.339449+0000","last_fullsized":"2024-09-12T02:48:37.339449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336705+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:48:37.336705+0000","last_peered":"2024-09-12T02:48:37.336705+0000","last_clean":"2024-09-12T02:48:37.336705+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:48:37.336705+0000","last_undegraded":"2024-09-12T02:48:37.336705+0000","last_fullsized":"2024-09-12T02:48:37.336705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341250+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:37.341250+0000","last_peered":"2024-09-12T02:48:37.341250+0000","last_clean":"2024-09-12T02:48:37.341250+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:37.341250+0000","last_undegraded":"2024-09-12T02:48:37.341250+0000","last_fullsized":"2024-09-12T02:48:37.341250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:48:37.341552+0000","last_peered":"2024-09-12T02:48:37.341552+0000","last_clean":"2024-09-12T02:48:37.341552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:48:37.341552+0000","last_undegraded":"2024-09-12T02:48:37.341552+0000","last_fullsized":"2024-09-12T02:48:37.341552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343136+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:48:37.343136+0000","last_peered":"2024-09-12T02:48:37.343136+0000","last_clean":"2024-09-12T02:48:37.343136+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:48:37.343136+0000","last_undegraded":"2024-09-12T02:48:37.343136+0000","last_fullsized":"2024-09-12T02:48:37.343136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587950+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:37.587950+0000","last_peered":"2024-09-12T02:48:37.587950+0000","last_clean":"2024-09-12T02:48:37.587950+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:37.587950+0000","last_undegraded":"2024-09-12T02:48:37.587950+0000","last_fullsized":"2024-09-12T02:48:37.587950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339541+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:48:37.339541+0000","last_peered":"2024-09-12T02:48:37.339541+0000","last_clean":"2024-09-12T02:48:37.339541+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:48:37.339541+0000","last_undegraded":"2024-09-12T02:48:37.339541+0000","last_fullsized":"2024-09-12T02:48:37.339541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588702+0000","last_change":"2024-09-12T02:48:37.588702+0000","last_active":"2024-09-12T02:48:37.588702+0000","last_peered":"2024-09-12T02:48:37.588702+0000","last_clean":"2024-09-12T02:48:37.588702+0000","last_became_active":"2024-09-12T02:48:37.588347+0000","last_became_peered":"2024-09-12T02:48:37.588347+0000","last_unstale":"2024-09-12T02:48:37.588702+0000","last_undegraded":"2024-09-12T02:48:37.588702+0000","last_fullsized":"2024-09-12T02:48:37.588702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:30.609070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586990+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:48:37.586990+0000","last_peered":"2024-09-12T02:48:37.586990+0000","last_clean":"2024-09-12T02:48:37.586990+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:48:37.586990+0000","last_undegraded":"2024-09-12T02:48:37.586990+0000","last_fullsized":"2024-09-12T02:48:37.586990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336464+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:48:37.336464+0000","last_peered":"2024-09-12T02:48:37.336464+0000","last_clean":"2024-09-12T02:48:37.336464+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:48:37.336464+0000","last_undegraded":"2024-09-12T02:48:37.336464+0000","last_fullsized":"2024-09-12T02:48:37.336464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337504+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:37.337504+0000","last_peered":"2024-09-12T02:48:37.337504+0000","last_clean":"2024-09-12T02:48:37.337504+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:37.337504+0000","last_undegraded":"2024-09-12T02:48:37.337504+0000","last_fullsized":"2024-09-12T02:48:37.337504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:43.138761+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:48:43.138761+0000","last_peered":"2024-09-12T02:48:43.138761+0000","last_clean":"2024-09-12T02:48:43.138761+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:48:43.138761+0000","last_undegraded":"2024-09-12T02:48:43.138761+0000","last_fullsized":"2024-09-12T02:48:43.138761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336272+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:48:37.336272+0000","last_peered":"2024-09-12T02:48:37.336272+0000","last_clean":"2024-09-12T02:48:37.336272+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:48:37.336272+0000","last_undegraded":"2024-09-12T02:48:37.336272+0000","last_fullsized":"2024-09-12T02:48:37.336272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588802+0000","last_change":"2024-09-12T02:48:37.588802+0000","last_active":"2024-09-12T02:48:37.588802+0000","last_peered":"2024-09-12T02:48:37.588802+0000","last_clean":"2024-09-12T02:48:37.588802+0000","last_became_active":"2024-09-12T02:48:37.588326+0000","last_became_peered":"2024-09-12T02:48:37.588326+0000","last_unstale":"2024-09-12T02:48:37.588802+0000","last_undegraded":"2024-09-12T02:48:37.588802+0000","last_fullsized":"2024-09-12T02:48:37.588802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586758+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:37.586758+0000","last_peered":"2024-09-12T02:48:37.586758+0000","last_clean":"2024-09-12T02:48:37.586758+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:37.586758+0000","last_undegraded":"2024-09-12T02:48:37.586758+0000","last_fullsized":"2024-09-12T02:48:37.586758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345128+0000","last_change":"2024-09-12T02:48:37.345128+0000","last_active":"2024-09-12T02:48:37.345128+0000","last_peered":"2024-09-12T02:48:37.345128+0000","last_clean":"2024-09-12T02:48:37.345128+0000","last_became_active":"2024-09-12T02:48:37.344670+0000","last_became_peered":"2024-09-12T02:48:37.344670+0000","last_unstale":"2024-09-12T02:48:37.345128+0000","last_undegraded":"2024-09-12T02:48:37.345128+0000","last_fullsized":"2024-09-12T02:48:37.345128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:32.206800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339329+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:48:37.339329+0000","last_peered":"2024-09-12T02:48:37.339329+0000","last_clean":"2024-09-12T02:48:37.339329+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:48:37.339329+0000","last_undegraded":"2024-09-12T02:48:37.339329+0000","last_fullsized":"2024-09-12T02:48:37.339329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341367+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:37.341367+0000","last_peered":"2024-09-12T02:48:37.341367+0000","last_clean":"2024-09-12T02:48:37.341367+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:37.341367+0000","last_undegraded":"2024-09-12T02:48:37.341367+0000","last_fullsized":"2024-09-12T02:48:37.341367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586906+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:48:37.586906+0000","last_peered":"2024-09-12T02:48:37.586906+0000","last_clean":"2024-09-12T02:48:37.586906+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:48:37.586906+0000","last_undegraded":"2024-09-12T02:48:37.586906+0000","last_fullsized":"2024-09-12T02:48:37.586906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586973+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:48:37.586973+0000","last_peered":"2024-09-12T02:48:37.586973+0000","last_clean":"2024-09-12T02:48:37.586973+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:48:37.586973+0000","last_undegraded":"2024-09-12T02:48:37.586973+0000","last_fullsized":"2024-09-12T02:48:37.586973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587180+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:48:37.587180+0000","last_peered":"2024-09-12T02:48:37.587180+0000","last_clean":"2024-09-12T02:48:37.587180+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:48:37.587180+0000","last_undegraded":"2024-09-12T02:48:37.587180+0000","last_fullsized":"2024-09-12T02:48:37.587180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343649+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:37.343649+0000","last_peered":"2024-09-12T02:48:37.343649+0000","last_clean":"2024-09-12T02:48:37.343649+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:37.343649+0000","last_undegraded":"2024-09-12T02:48:37.343649+0000","last_fullsized":"2024-09-12T02:48:37.343649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339501+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:48:37.339501+0000","last_peered":"2024-09-12T02:48:37.339501+0000","last_clean":"2024-09-12T02:48:37.339501+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:48:37.339501+0000","last_undegraded":"2024-09-12T02:48:37.339501+0000","last_fullsized":"2024-09-12T02:48:37.339501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587059+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:48:37.587059+0000","last_peered":"2024-09-12T02:48:37.587059+0000","last_clean":"2024-09-12T02:48:37.587059+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:48:37.587059+0000","last_undegraded":"2024-09-12T02:48:37.587059+0000","last_fullsized":"2024-09-12T02:48:37.587059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338318+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:37.338318+0000","last_peered":"2024-09-12T02:48:37.338318+0000","last_clean":"2024-09-12T02:48:37.338318+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:37.338318+0000","last_undegraded":"2024-09-12T02:48:37.338318+0000","last_fullsized":"2024-09-12T02:48:37.338318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.346098+0000","last_change":"2024-09-12T02:48:37.346098+0000","last_active":"2024-09-12T02:48:37.346098+0000","last_peered":"2024-09-12T02:48:37.346098+0000","last_clean":"2024-09-12T02:48:37.346098+0000","last_became_active":"2024-09-12T02:48:37.345696+0000","last_became_peered":"2024-09-12T02:48:37.345696+0000","last_unstale":"2024-09-12T02:48:37.346098+0000","last_undegraded":"2024-09-12T02:48:37.346098+0000","last_fullsized":"2024-09-12T02:48:37.346098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:48.040441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587347+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:48:37.587347+0000","last_peered":"2024-09-12T02:48:37.587347+0000","last_clean":"2024-09-12T02:48:37.587347+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:48:37.587347+0000","last_undegraded":"2024-09-12T02:48:37.587347+0000","last_fullsized":"2024-09-12T02:48:37.587347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336481+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:48:37.336481+0000","last_peered":"2024-09-12T02:48:37.336481+0000","last_clean":"2024-09-12T02:48:37.336481+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:48:37.336481+0000","last_undegraded":"2024-09-12T02:48:37.336481+0000","last_fullsized":"2024-09-12T02:48:37.336481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337296+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:37.337296+0000","last_peered":"2024-09-12T02:48:37.337296+0000","last_clean":"2024-09-12T02:48:37.337296+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:37.337296+0000","last_undegraded":"2024-09-12T02:48:37.337296+0000","last_fullsized":"2024-09-12T02:48:37.337296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341055+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:48:37.341055+0000","last_peered":"2024-09-12T02:48:37.341055+0000","last_clean":"2024-09-12T02:48:37.341055+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:48:37.341055+0000","last_undegraded":"2024-09-12T02:48:37.341055+0000","last_fullsized":"2024-09-12T02:48:37.341055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340312+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:37.340312+0000","last_peered":"2024-09-12T02:48:37.340312+0000","last_clean":"2024-09-12T02:48:37.340312+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:37.340312+0000","last_undegraded":"2024-09-12T02:48:37.340312+0000","last_fullsized":"2024-09-12T02:48:37.340312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337321+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:37.337321+0000","last_peered":"2024-09-12T02:48:37.337321+0000","last_clean":"2024-09-12T02:48:37.337321+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:37.337321+0000","last_undegraded":"2024-09-12T02:48:37.337321+0000","last_fullsized":"2024-09-12T02:48:37.337321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337404+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:37.337404+0000","last_peered":"2024-09-12T02:48:37.337404+0000","last_clean":"2024-09-12T02:48:37.337404+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:37.337404+0000","last_undegraded":"2024-09-12T02:48:37.337404+0000","last_fullsized":"2024-09-12T02:48:37.337404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339395+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:37.339395+0000","last_peered":"2024-09-12T02:48:37.339395+0000","last_clean":"2024-09-12T02:48:37.339395+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:37.339395+0000","last_undegraded":"2024-09-12T02:48:37.339395+0000","last_fullsized":"2024-09-12T02:48:37.339395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.344145+0000","last_change":"2024-09-12T02:48:37.344145+0000","last_active":"2024-09-12T02:48:37.344145+0000","last_peered":"2024-09-12T02:48:37.344145+0000","last_clean":"2024-09-12T02:48:37.344145+0000","last_became_active":"2024-09-12T02:48:37.343694+0000","last_became_peered":"2024-09-12T02:48:37.343694+0000","last_unstale":"2024-09-12T02:48:37.344145+0000","last_undegraded":"2024-09-12T02:48:37.344145+0000","last_fullsized":"2024-09-12T02:48:37.344145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:55.742228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586644+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:48:37.586644+0000","last_peered":"2024-09-12T02:48:37.586644+0000","last_clean":"2024-09-12T02:48:37.586644+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:48:37.586644+0000","last_undegraded":"2024-09-12T02:48:37.586644+0000","last_fullsized":"2024-09-12T02:48:37.586644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:45.142424+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:48:45.142424+0000","last_peered":"2024-09-12T02:48:45.142424+0000","last_clean":"2024-09-12T02:48:45.142424+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:48:45.142424+0000","last_undegraded":"2024-09-12T02:48:45.142424+0000","last_fullsized":"2024-09-12T02:48:45.142424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587962+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:37.587962+0000","last_peered":"2024-09-12T02:48:37.587962+0000","last_clean":"2024-09-12T02:48:37.587962+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:37.587962+0000","last_undegraded":"2024-09-12T02:48:37.587962+0000","last_fullsized":"2024-09-12T02:48:37.587962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587055+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:48:37.587055+0000","last_peered":"2024-09-12T02:48:37.587055+0000","last_clean":"2024-09-12T02:48:37.587055+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:48:37.587055+0000","last_undegraded":"2024-09-12T02:48:37.587055+0000","last_fullsized":"2024-09-12T02:48:37.587055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337741+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:37.337741+0000","last_peered":"2024-09-12T02:48:37.337741+0000","last_clean":"2024-09-12T02:48:37.337741+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:37.337741+0000","last_undegraded":"2024-09-12T02:48:37.337741+0000","last_fullsized":"2024-09-12T02:48:37.337741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341156+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:48:37.341156+0000","last_peered":"2024-09-12T02:48:37.341156+0000","last_clean":"2024-09-12T02:48:37.341156+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:48:37.341156+0000","last_undegraded":"2024-09-12T02:48:37.341156+0000","last_fullsized":"2024-09-12T02:48:37.341156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336133+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:48:37.336133+0000","last_peered":"2024-09-12T02:48:37.336133+0000","last_clean":"2024-09-12T02:48:37.336133+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:48:37.336133+0000","last_undegraded":"2024-09-12T02:48:37.336133+0000","last_fullsized":"2024-09-12T02:48:37.336133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.353056+0000","last_change":"2024-09-12T02:48:37.352674+0000","last_active":"2024-09-12T02:48:37.353056+0000","last_peered":"2024-09-12T02:48:37.353056+0000","last_clean":"2024-09-12T02:48:37.353056+0000","last_became_active":"2024-09-12T02:48:37.345293+0000","last_became_peered":"2024-09-12T02:48:37.345293+0000","last_unstale":"2024-09-12T02:48:37.353056+0000","last_undegraded":"2024-09-12T02:48:37.353056+0000","last_fullsized":"2024-09-12T02:48:37.353056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:00.609598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338121+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:37.338121+0000","last_peered":"2024-09-12T02:48:37.338121+0000","last_clean":"2024-09-12T02:48:37.338121+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:37.338121+0000","last_undegraded":"2024-09-12T02:48:37.338121+0000","last_fullsized":"2024-09-12T02:48:37.338121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340161+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:48:37.340161+0000","last_peered":"2024-09-12T02:48:37.340161+0000","last_clean":"2024-09-12T02:48:37.340161+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:48:37.340161+0000","last_undegraded":"2024-09-12T02:48:37.340161+0000","last_fullsized":"2024-09-12T02:48:37.340161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":131,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:46.147086+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:48:46.147086+0000","last_peered":"2024-09-12T02:48:46.147086+0000","last_clean":"2024-09-12T02:48:46.147086+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:48:46.147086+0000","last_undegraded":"2024-09-12T02:48:46.147086+0000","last_fullsized":"2024-09-12T02:48:46.147086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":63,"num_read_kb":94,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587651+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:37.587651+0000","last_peered":"2024-09-12T02:48:37.587651+0000","last_clean":"2024-09-12T02:48:37.587651+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:37.587651+0000","last_undegraded":"2024-09-12T02:48:37.587651+0000","last_fullsized":"2024-09-12T02:48:37.587651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339610+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:48:37.339610+0000","last_peered":"2024-09-12T02:48:37.339610+0000","last_clean":"2024-09-12T02:48:37.339610+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:48:37.339610+0000","last_undegraded":"2024-09-12T02:48:37.339610+0000","last_fullsized":"2024-09-12T02:48:37.339610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338294+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:37.338294+0000","last_peered":"2024-09-12T02:48:37.338294+0000","last_clean":"2024-09-12T02:48:37.338294+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:37.338294+0000","last_undegraded":"2024-09-12T02:48:37.338294+0000","last_fullsized":"2024-09-12T02:48:37.338294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341022+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:37.341022+0000","last_peered":"2024-09-12T02:48:37.341022+0000","last_clean":"2024-09-12T02:48:37.341022+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:37.341022+0000","last_undegraded":"2024-09-12T02:48:37.341022+0000","last_fullsized":"2024-09-12T02:48:37.341022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341169+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:48:37.341169+0000","last_peered":"2024-09-12T02:48:37.341169+0000","last_clean":"2024-09-12T02:48:37.341169+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:48:37.341169+0000","last_undegraded":"2024-09-12T02:48:37.341169+0000","last_fullsized":"2024-09-12T02:48:37.341169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587207+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:37.587207+0000","last_peered":"2024-09-12T02:48:37.587207+0000","last_clean":"2024-09-12T02:48:37.587207+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:37.587207+0000","last_undegraded":"2024-09-12T02:48:37.587207+0000","last_fullsized":"2024-09-12T02:48:37.587207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587311+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:37.587311+0000","last_peered":"2024-09-12T02:48:37.587311+0000","last_clean":"2024-09-12T02:48:37.587311+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:37.587311+0000","last_undegraded":"2024-09-12T02:48:37.587311+0000","last_fullsized":"2024-09-12T02:48:37.587311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340162+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:48:37.340162+0000","last_peered":"2024-09-12T02:48:37.340162+0000","last_clean":"2024-09-12T02:48:37.340162+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:48:37.340162+0000","last_undegraded":"2024-09-12T02:48:37.340162+0000","last_fullsized":"2024-09-12T02:48:37.340162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:45.141017+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:48:45.141017+0000","last_peered":"2024-09-12T02:48:45.141017+0000","last_clean":"2024-09-12T02:48:45.141017+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:48:45.141017+0000","last_undegraded":"2024-09-12T02:48:45.141017+0000","last_fullsized":"2024-09-12T02:48:45.141017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339163+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:48:37.339163+0000","last_peered":"2024-09-12T02:48:37.339163+0000","last_clean":"2024-09-12T02:48:37.339163+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:48:37.339163+0000","last_undegraded":"2024-09-12T02:48:37.339163+0000","last_fullsized":"2024-09-12T02:48:37.339163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340919+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:48:37.340919+0000","last_peered":"2024-09-12T02:48:37.340919+0000","last_clean":"2024-09-12T02:48:37.340919+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:48:37.340919+0000","last_undegraded":"2024-09-12T02:48:37.340919+0000","last_fullsized":"2024-09-12T02:48:37.340919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587478+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:37.587478+0000","last_peered":"2024-09-12T02:48:37.587478+0000","last_clean":"2024-09-12T02:48:37.587478+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:37.587478+0000","last_undegraded":"2024-09-12T02:48:37.587478+0000","last_fullsized":"2024-09-12T02:48:37.587478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337681+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:37.337681+0000","last_peered":"2024-09-12T02:48:37.337681+0000","last_clean":"2024-09-12T02:48:37.337681+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:37.337681+0000","last_undegraded":"2024-09-12T02:48:37.337681+0000","last_fullsized":"2024-09-12T02:48:37.337681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340797+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:48:37.340797+0000","last_peered":"2024-09-12T02:48:37.340797+0000","last_clean":"2024-09-12T02:48:37.340797+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:48:37.340797+0000","last_undegraded":"2024-09-12T02:48:37.340797+0000","last_fullsized":"2024-09-12T02:48:37.340797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339975+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:48:37.339975+0000","last_peered":"2024-09-12T02:48:37.339975+0000","last_clean":"2024-09-12T02:48:37.339975+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:48:37.339975+0000","last_undegraded":"2024-09-12T02:48:37.339975+0000","last_fullsized":"2024-09-12T02:48:37.339975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336024+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:37.336024+0000","last_peered":"2024-09-12T02:48:37.336024+0000","last_clean":"2024-09-12T02:48:37.336024+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:37.336024+0000","last_undegraded":"2024-09-12T02:48:37.336024+0000","last_fullsized":"2024-09-12T02:48:37.336024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343029+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:37.343029+0000","last_peered":"2024-09-12T02:48:37.343029+0000","last_clean":"2024-09-12T02:48:37.343029+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:37.343029+0000","last_undegraded":"2024-09-12T02:48:37.343029+0000","last_fullsized":"2024-09-12T02:48:37.343029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337742+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:37.337742+0000","last_peered":"2024-09-12T02:48:37.337742+0000","last_clean":"2024-09-12T02:48:37.337742+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:37.337742+0000","last_undegraded":"2024-09-12T02:48:37.337742+0000","last_fullsized":"2024-09-12T02:48:37.337742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343312+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:48:37.343312+0000","last_peered":"2024-09-12T02:48:37.343312+0000","last_clean":"2024-09-12T02:48:37.343312+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:48:37.343312+0000","last_undegraded":"2024-09-12T02:48:37.343312+0000","last_fullsized":"2024-09-12T02:48:37.343312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:46.142996+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:48:46.142996+0000","last_peered":"2024-09-12T02:48:46.142996+0000","last_clean":"2024-09-12T02:48:46.142996+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:48:46.142996+0000","last_undegraded":"2024-09-12T02:48:46.142996+0000","last_fullsized":"2024-09-12T02:48:46.142996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588956+0000","last_change":"2024-09-12T02:48:37.588956+0000","last_active":"2024-09-12T02:48:37.588956+0000","last_peered":"2024-09-12T02:48:37.588956+0000","last_clean":"2024-09-12T02:48:37.588956+0000","last_became_active":"2024-09-12T02:48:37.588561+0000","last_became_peered":"2024-09-12T02:48:37.588561+0000","last_unstale":"2024-09-12T02:48:37.588956+0000","last_undegraded":"2024-09-12T02:48:37.588956+0000","last_fullsized":"2024-09-12T02:48:37.588956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:08.645076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343699+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:48:37.343699+0000","last_peered":"2024-09-12T02:48:37.343699+0000","last_clean":"2024-09-12T02:48:37.343699+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:48:37.343699+0000","last_undegraded":"2024-09-12T02:48:37.343699+0000","last_fullsized":"2024-09-12T02:48:37.343699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340641+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:48:37.340641+0000","last_peered":"2024-09-12T02:48:37.340641+0000","last_clean":"2024-09-12T02:48:37.340641+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:48:37.340641+0000","last_undegraded":"2024-09-12T02:48:37.340641+0000","last_fullsized":"2024-09-12T02:48:37.340641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343329+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:48:37.343329+0000","last_peered":"2024-09-12T02:48:37.343329+0000","last_clean":"2024-09-12T02:48:37.343329+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:48:37.343329+0000","last_undegraded":"2024-09-12T02:48:37.343329+0000","last_fullsized":"2024-09-12T02:48:37.343329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.589056+0000","last_change":"2024-09-12T02:48:37.589056+0000","last_active":"2024-09-12T02:48:37.589056+0000","last_peered":"2024-09-12T02:48:37.589056+0000","last_clean":"2024-09-12T02:48:37.589056+0000","last_became_active":"2024-09-12T02:48:37.588629+0000","last_became_peered":"2024-09-12T02:48:37.588629+0000","last_unstale":"2024-09-12T02:48:37.589056+0000","last_undegraded":"2024-09-12T02:48:37.589056+0000","last_fullsized":"2024-09-12T02:48:37.589056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:03.490341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340249+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:48:37.340249+0000","last_peered":"2024-09-12T02:48:37.340249+0000","last_clean":"2024-09-12T02:48:37.340249+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:48:37.340249+0000","last_undegraded":"2024-09-12T02:48:37.340249+0000","last_fullsized":"2024-09-12T02:48:37.340249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336590+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:48:37.336590+0000","last_peered":"2024-09-12T02:48:37.336590+0000","last_clean":"2024-09-12T02:48:37.336590+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:48:37.336590+0000","last_undegraded":"2024-09-12T02:48:37.336590+0000","last_fullsized":"2024-09-12T02:48:37.336590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341713+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:48:37.341713+0000","last_peered":"2024-09-12T02:48:37.341713+0000","last_clean":"2024-09-12T02:48:37.341713+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:48:37.341713+0000","last_undegraded":"2024-09-12T02:48:37.341713+0000","last_fullsized":"2024-09-12T02:48:37.341713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343987+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:48:37.343987+0000","last_peered":"2024-09-12T02:48:37.343987+0000","last_clean":"2024-09-12T02:48:37.343987+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:48:37.343987+0000","last_undegraded":"2024-09-12T02:48:37.343987+0000","last_fullsized":"2024-09-12T02:48:37.343987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345556+0000","last_change":"2024-09-12T02:48:37.345556+0000","last_active":"2024-09-12T02:48:37.345556+0000","last_peered":"2024-09-12T02:48:37.345556+0000","last_clean":"2024-09-12T02:48:37.345556+0000","last_became_active":"2024-09-12T02:48:37.345220+0000","last_became_peered":"2024-09-12T02:48:37.345220+0000","last_unstale":"2024-09-12T02:48:37.345556+0000","last_undegraded":"2024-09-12T02:48:37.345556+0000","last_fullsized":"2024-09-12T02:48:37.345556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:25.562372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337339+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:37.337339+0000","last_peered":"2024-09-12T02:48:37.337339+0000","last_clean":"2024-09-12T02:48:37.337339+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:37.337339+0000","last_undegraded":"2024-09-12T02:48:37.337339+0000","last_fullsized":"2024-09-12T02:48:37.337339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339282+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:48:37.339282+0000","last_peered":"2024-09-12T02:48:37.339282+0000","last_clean":"2024-09-12T02:48:37.339282+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:48:37.339282+0000","last_undegraded":"2024-09-12T02:48:37.339282+0000","last_fullsized":"2024-09-12T02:48:37.339282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339814+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:48:37.339814+0000","last_peered":"2024-09-12T02:48:37.339814+0000","last_clean":"2024-09-12T02:48:37.339814+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:48:37.339814+0000","last_undegraded":"2024-09-12T02:48:37.339814+0000","last_fullsized":"2024-09-12T02:48:37.339814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336265+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:48:37.336265+0000","last_peered":"2024-09-12T02:48:37.336265+0000","last_clean":"2024-09-12T02:48:37.336265+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:48:37.336265+0000","last_undegraded":"2024-09-12T02:48:37.336265+0000","last_fullsized":"2024-09-12T02:48:37.336265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341263+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:48:37.341263+0000","last_peered":"2024-09-12T02:48:37.341263+0000","last_clean":"2024-09-12T02:48:37.341263+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:48:37.341263+0000","last_undegraded":"2024-09-12T02:48:37.341263+0000","last_fullsized":"2024-09-12T02:48:37.341263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343830+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:48:37.343830+0000","last_peered":"2024-09-12T02:48:37.343830+0000","last_clean":"2024-09-12T02:48:37.343830+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:48:37.343830+0000","last_undegraded":"2024-09-12T02:48:37.343830+0000","last_fullsized":"2024-09-12T02:48:37.343830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588060+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:37.588060+0000","last_peered":"2024-09-12T02:48:37.588060+0000","last_clean":"2024-09-12T02:48:37.588060+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:37.588060+0000","last_undegraded":"2024-09-12T02:48:37.588060+0000","last_fullsized":"2024-09-12T02:48:37.588060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342161+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:48:37.342161+0000","last_peered":"2024-09-12T02:48:37.342161+0000","last_clean":"2024-09-12T02:48:37.342161+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:48:37.342161+0000","last_undegraded":"2024-09-12T02:48:37.342161+0000","last_fullsized":"2024-09-12T02:48:37.342161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339233+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:48:37.339233+0000","last_peered":"2024-09-12T02:48:37.339233+0000","last_clean":"2024-09-12T02:48:37.339233+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:48:37.339233+0000","last_undegraded":"2024-09-12T02:48:37.339233+0000","last_fullsized":"2024-09-12T02:48:37.339233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339694+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:48:37.339694+0000","last_peered":"2024-09-12T02:48:37.339694+0000","last_clean":"2024-09-12T02:48:37.339694+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:48:37.339694+0000","last_undegraded":"2024-09-12T02:48:37.339694+0000","last_fullsized":"2024-09-12T02:48:37.339694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339992+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:48:37.339992+0000","last_peered":"2024-09-12T02:48:37.339992+0000","last_clean":"2024-09-12T02:48:37.339992+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:48:37.339992+0000","last_undegraded":"2024-09-12T02:48:37.339992+0000","last_fullsized":"2024-09-12T02:48:37.339992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336229+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:48:37.336229+0000","last_peered":"2024-09-12T02:48:37.336229+0000","last_clean":"2024-09-12T02:48:37.336229+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:48:37.336229+0000","last_undegraded":"2024-09-12T02:48:37.336229+0000","last_fullsized":"2024-09-12T02:48:37.336229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341015+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:37.341015+0000","last_peered":"2024-09-12T02:48:37.341015+0000","last_clean":"2024-09-12T02:48:37.341015+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:37.341015+0000","last_undegraded":"2024-09-12T02:48:37.341015+0000","last_fullsized":"2024-09-12T02:48:37.341015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587114+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:48:37.587114+0000","last_peered":"2024-09-12T02:48:37.587114+0000","last_clean":"2024-09-12T02:48:37.587114+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:48:37.587114+0000","last_undegraded":"2024-09-12T02:48:37.587114+0000","last_fullsized":"2024-09-12T02:48:37.587114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343668+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:48:37.343668+0000","last_peered":"2024-09-12T02:48:37.343668+0000","last_clean":"2024-09-12T02:48:37.343668+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:48:37.343668+0000","last_undegraded":"2024-09-12T02:48:37.343668+0000","last_fullsized":"2024-09-12T02:48:37.343668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340009+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:48:37.340009+0000","last_peered":"2024-09-12T02:48:37.340009+0000","last_clean":"2024-09-12T02:48:37.340009+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:48:37.340009+0000","last_undegraded":"2024-09-12T02:48:37.340009+0000","last_fullsized":"2024-09-12T02:48:37.340009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339344+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:48:37.339344+0000","last_peered":"2024-09-12T02:48:37.339344+0000","last_clean":"2024-09-12T02:48:37.339344+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:48:37.339344+0000","last_undegraded":"2024-09-12T02:48:37.339344+0000","last_fullsized":"2024-09-12T02:48:37.339344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586757+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:48:37.586757+0000","last_peered":"2024-09-12T02:48:37.586757+0000","last_clean":"2024-09-12T02:48:37.586757+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:48:37.586757+0000","last_undegraded":"2024-09-12T02:48:37.586757+0000","last_fullsized":"2024-09-12T02:48:37.586757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342802+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:48:37.342802+0000","last_peered":"2024-09-12T02:48:37.342802+0000","last_clean":"2024-09-12T02:48:37.342802+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:48:37.342802+0000","last_undegraded":"2024-09-12T02:48:37.342802+0000","last_fullsized":"2024-09-12T02:48:37.342802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339808+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:48:37.339808+0000","last_peered":"2024-09-12T02:48:37.339808+0000","last_clean":"2024-09-12T02:48:37.339808+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:48:37.339808+0000","last_undegraded":"2024-09-12T02:48:37.339808+0000","last_fullsized":"2024-09-12T02:48:37.339808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340891+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:37.340891+0000","last_peered":"2024-09-12T02:48:37.340891+0000","last_clean":"2024-09-12T02:48:37.340891+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:37.340891+0000","last_undegraded":"2024-09-12T02:48:37.340891+0000","last_fullsized":"2024-09-12T02:48:37.340891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340897+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:48:37.340897+0000","last_peered":"2024-09-12T02:48:37.340897+0000","last_clean":"2024-09-12T02:48:37.340897+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:48:37.340897+0000","last_undegraded":"2024-09-12T02:48:37.340897+0000","last_fullsized":"2024-09-12T02:48:37.340897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341040+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:48:37.341040+0000","last_peered":"2024-09-12T02:48:37.341040+0000","last_clean":"2024-09-12T02:48:37.341040+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:48:37.341040+0000","last_undegraded":"2024-09-12T02:48:37.341040+0000","last_fullsized":"2024-09-12T02:48:37.341040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342054+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:48:37.342054+0000","last_peered":"2024-09-12T02:48:37.342054+0000","last_clean":"2024-09-12T02:48:37.342054+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:48:37.342054+0000","last_undegraded":"2024-09-12T02:48:37.342054+0000","last_fullsized":"2024-09-12T02:48:37.342054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":189,"num_read_kb":165,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":501459,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743455,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1152757,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":505757,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135914,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":501581,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":1097492,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528371,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":496837,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1056768,"data_stored":1093080,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":501487,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":497031,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":562520,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":560811,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":498225,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:48:49.382 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:48:49.383 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T02:48:49.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T02:48:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- 172.21.15.46:0/575131650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 msgr2=0x7f7d04107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/575131650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04107380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009900 tx=0x7f7cf002a410 comp rx=0 tx=0).stop 2024-09-12T02:48:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- 172.21.15.46:0/575131650 shutdown_connections 2024-09-12T02:48:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/575131650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/575131650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04107380 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/575131650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 0x7f7d04104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- 172.21.15.46:0/575131650 >> 172.21.15.46:0/575131650 conn(0x7f7d040fc280 msgr2=0x7f7d040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- 172.21.15.46:0/575131650 shutdown_connections 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- 172.21.15.46:0/575131650 wait complete. 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 Processor -- start 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.144+0000 7f7d0ab75640 1 -- start start 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 0x7f7d04101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410d710 con 0x7f7d04104f90 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410d590 con 0x7f7d04102660 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410d890 con 0x7f7d041078c0 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34752/0 (socket says 172.21.15.46:34752) 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 -- 172.21.15.46:0/2414494407 learned_addr learned my addr 172.21.15.46:0/2414494407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d08911640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 0x7f7d04101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d09112640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 -- 172.21.15.46:0/2414494407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 msgr2=0x7f7d04100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 -- 172.21.15.46:0/2414494407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 msgr2=0x7f7d04101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 0x7f7d04101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 -- 172.21.15.46:0/2414494407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04101410 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d09112640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d03fff640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009230 tx=0x7f7cf0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf003f030 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041a1f00 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cf0003cb0 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.145+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0035070 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.146+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a2300 con 0x7f7d04104f90 2024-09-12T02:48:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.147+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d040fdd70 con 0x7f7d04104f90 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.148+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7cf0042030 con 0x7f7d04104f90 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.149+0000 7f7d01ffb640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 0x7f7cf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.149+0000 7f7d08911640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 0x7f7cf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.149+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7cf00effa0 con 0x7f7d04104f90 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.149+0000 7f7d08911640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 0x7f7cf40a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800abe0 tx=0x7f7cf800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.152+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cf00bca40 con 0x7f7d04104f90 2024-09-12T02:48:50.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.394+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7d0402d7c0 con 0x7f7d04104f90 2024-09-12T02:48:50.396 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:50 smithi046 ceph-mon[30951]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:50.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:50 smithi046 ceph-mon[30951]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.397+0000 7f7d01ffb640 1 -- 172.21.15.46:0/2414494407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7cf00bc340 con 0x7f7d04104f90 2024-09-12T02:48:50.398 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-12T02:48:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSVuJmt6eoFxAATxH40Qn5993Q/w0rHfaCFA== 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 msgr2=0x7f7cf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 0x7f7cf40a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800abe0 tx=0x7f7cf800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 msgr2=0x7f7d04100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009230 tx=0x7f7cf0004340 comp rx=0 tx=0).stop 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 shutdown_connections 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f7cf40a2c10 0x7f7cf40a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d041078c0 0x7f7d04100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04104f90 0x7f7d04100610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.400+0000 7f7d0ab75640 1 --2- 172.21.15.46:0/2414494407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d04102660 0x7f7d04101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.401+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 >> 172.21.15.46:0/2414494407 conn(0x7f7d040fc280 msgr2=0x7f7d04102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.401+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 shutdown_connections 2024-09-12T02:48:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:50.401+0000 7f7d0ab75640 1 -- 172.21.15.46:0/2414494407 wait complete. 2024-09-12T02:48:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:50 smithi196 ceph-mon[34587]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:50 smithi196 ceph-mon[34587]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:50 smithi145 ceph-mon[34674]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:50 smithi145 ceph-mon[34674]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:48:51.045 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:48:51.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T02:48:51.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T02:48:51.085 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T02:48:51.085 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T02:48:51.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T02:48:51.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:51 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:48:51.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:51 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:48:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:51 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:48:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:51 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:48:51.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:51 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:48:51.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:51 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2414494407' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 -- 172.21.15.46:0/965129689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 msgr2=0x7f3430100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 --2- 172.21.15.46:0/965129689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f3430100ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3418009930 tx=0x7f341802a420 comp rx=0 tx=0).stop 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 -- 172.21.15.46:0/965129689 shutdown_connections 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 --2- 172.21.15.46:0/965129689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430101420 0x7f343010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 --2- 172.21.15.46:0/965129689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f3430100ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 --2- 172.21.15.46:0/965129689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430106a80 0x7f3430106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 -- 172.21.15.46:0/965129689 >> 172.21.15.46:0/965129689 conn(0x7f34300fc7a0 msgr2=0x7f34300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 -- 172.21.15.46:0/965129689 shutdown_connections 2024-09-12T02:48:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.867+0000 7f343594a640 1 -- 172.21.15.46:0/965129689 wait complete. 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 Processor -- start 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 -- start start 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430101420 0x7f34300fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430106a80 0x7f34300fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3430110d30 con 0x7f3430100a80 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3430110bb0 con 0x7f3430106a80 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f343594a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3430110eb0 con 0x7f3430101420 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f342effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.868+0000 7f342effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34774/0 (socket says 172.21.15.46:34774) 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 -- 172.21.15.46:0/1331931232 learned_addr learned my addr 172.21.15.46:0/1331931232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f34267fc640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430101420 0x7f34300fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342f7fe640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430106a80 0x7f34300fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 -- 172.21.15.46:0/1331931232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430101420 msgr2=0x7f34300fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430101420 0x7f34300fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 -- 172.21.15.46:0/1331931232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430106a80 msgr2=0x7f34300fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430106a80 0x7f34300fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 -- 172.21.15.46:0/1331931232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34301a6820 con 0x7f3430100a80 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f342effd640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f342000d950 tx=0x7f342000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420017030 con 0x7f3430100a80 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3420004020 con 0x7f3430100a80 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34301a6b10 con 0x7f3430100a80 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34200047a0 con 0x7f3430100a80 2024-09-12T02:48:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.869+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34301a7020 con 0x7f3430100a80 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.870+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3430100ee0 con 0x7f3430100a80 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.871+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f34200041d0 con 0x7f3430100a80 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.872+0000 7f3434948640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 0x7f341c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.872+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f34200cf5b0 con 0x7f3430100a80 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.872+0000 7f34267fc640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 0x7f341c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.873+0000 7f34267fc640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 0x7f341c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f3418000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:51.875+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f342009c050 con 0x7f3430100a80 2024-09-12T02:48:52.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.096+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f34300fe360 con 0x7f3430100a80 2024-09-12T02:48:52.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.101+0000 7f3434948640 1 -- 172.21.15.46:0/1331931232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f34300fe360 con 0x7f3430100a80 2024-09-12T02:48:52.105 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 msgr2=0x7f341c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 0x7f341c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f3418000bc0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 msgr2=0x7f34300fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f342000d950 tx=0x7f342000de10 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 shutdown_connections 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3430106a80 0x7f34300fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f341c0a2c10 0x7f341c0a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3430101420 0x7f34300fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 --2- 172.21.15.46:0/1331931232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3430100a80 0x7f34300fce30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 >> 172.21.15.46:0/1331931232 conn(0x7f34300fc7a0 msgr2=0x7f343006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 shutdown_connections 2024-09-12T02:48:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:52.104+0000 7f343594a640 1 -- 172.21.15.46:0/1331931232 wait complete. 2024-09-12T02:48:52.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:52 smithi196 ceph-mon[34587]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:52.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:52 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1331931232' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:48:52.529 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:52 smithi046 ceph-mon[30951]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:52.529 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:52 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1331931232' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:48:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:52 smithi145 ceph-mon[34674]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T02:48:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:52 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1331931232' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:48:52.821 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3748867549},{"type":"v1","addr":"172.21.15.46:6801","nonce":3748867549}]},"active_addr":"172.21.15.46:6801/3748867549","active_change":"2024-09-12T02:40:36.158001+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2520496515}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":328833784}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2008665137}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1022575768}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":4065010278}]}]} 2024-09-12T02:48:52.827 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T02:48:52.828 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T02:48:52.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:48:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/152806669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8106f10 msgr2=0x7ff0c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/152806669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8106f10 0x7ff0c810ea90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009ca0 tx=0x7ff0c002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/152806669 shutdown_connections 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/152806669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8106f10 0x7ff0c810ea90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/152806669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8102790 0x7ff0c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/152806669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8101de0 0x7ff0c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.592+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/152806669 >> 172.21.15.46:0/152806669 conn(0x7ff0c80fd8c0 msgr2=0x7ff0c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/152806669 shutdown_connections 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/152806669 wait complete. 2024-09-12T02:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 Processor -- start 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- start start 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8102790 0x7ff0c81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8106f10 0x7ff0c819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8111e20 con 0x7ff0c8101de0 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8111ca0 con 0x7ff0c8102790 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cf8ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8111fa0 con 0x7ff0c8106f10 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.593+0000 7ff0cd649640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34784/0 (socket says 172.21.15.46:34784) 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 -- 172.21.15.46:0/3404337507 learned_addr learned my addr 172.21.15.46:0/3404337507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cde4a640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8106f10 0x7ff0c819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cce48640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8102790 0x7ff0c81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 -- 172.21.15.46:0/3404337507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8106f10 msgr2=0x7ff0c819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8106f10 0x7ff0c819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 -- 172.21.15.46:0/3404337507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8102790 msgr2=0x7ff0c81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8102790 0x7ff0c81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 -- 172.21.15.46:0/3404337507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c819d250 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cd649640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c920 tx=0x7ff0c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007c70 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c819d540 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c4007e10 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4018820 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.594+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c81a4e20 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.595+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c81021c0 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.596+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff0c40189c0 con 0x7ff0c8101de0 2024-09-12T02:48:53.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.597+0000 7ff0be7fc640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 0x7ff0b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:53.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.598+0000 7ff0cce48640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 0x7ff0b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:53.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.598+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff0c40cf6c0 con 0x7ff0c8101de0 2024-09-12T02:48:53.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.598+0000 7ff0cce48640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 0x7ff0b40a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005fd0 tx=0x7ff0b800b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:53.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.602+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0c4014030 con 0x7ff0c8101de0 2024-09-12T02:48:53.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.776+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0c8109b60 con 0x7ff0c8101de0 2024-09-12T02:48:53.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.778+0000 7ff0be7fc640 1 -- 172.21.15.46:0/3404337507 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24656 (secure 0 0 0) 0x7ff0c409c100 con 0x7ff0c8101de0 2024-09-12T02:48:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:53.780 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:48:37.333173+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:48:53.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 msgr2=0x7ff0b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:53.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 0x7ff0b40a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005fd0 tx=0x7ff0b800b510 comp rx=0 tx=0).stop 2024-09-12T02:48:53.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 msgr2=0x7ff0c81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:53.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c920 tx=0x7ff0c400cde0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 shutdown_connections 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff0b40a2c10 0x7ff0b40a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0c8106f10 0x7ff0c819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0c8102790 0x7ff0c81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 --2- 172.21.15.46:0/3404337507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0c8101de0 0x7ff0c81982a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 >> 172.21.15.46:0/3404337507 conn(0x7ff0c80fd8c0 msgr2=0x7ff0c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 shutdown_connections 2024-09-12T02:48:53.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:53.781+0000 7ff0cf8ad640 1 -- 172.21.15.46:0/3404337507 wait complete. 2024-09-12T02:48:54.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:54 smithi046 ceph-mon[30951]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:48:54.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:54 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3404337507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:54.467 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T02:48:54.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:48:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:54 smithi145 ceph-mon[34674]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:48:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:54 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3404337507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:54 smithi196 ceph-mon[34587]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:48:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:54 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3404337507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:54.731 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- 172.21.15.46:0/1593499044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70106f10 msgr2=0x7f1c7010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 --2- 172.21.15.46:0/1593499044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70106f10 0x7f1c7010ea90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ca0 tx=0x7f1c6802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- 172.21.15.46:0/1593499044 shutdown_connections 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 --2- 172.21.15.46:0/1593499044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70106f10 0x7f1c7010ea90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 --2- 172.21.15.46:0/1593499044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70102790 0x7f1c701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 --2- 172.21.15.46:0/1593499044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70101de0 0x7f1c701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- 172.21.15.46:0/1593499044 >> 172.21.15.46:0/1593499044 conn(0x7f1c700fd8c0 msgr2=0x7f1c700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- 172.21.15.46:0/1593499044 shutdown_connections 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- 172.21.15.46:0/1593499044 wait complete. 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 Processor -- start 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 -- start start 2024-09-12T02:48:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.211+0000 7f1c76a56640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70102790 0x7f1c701988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 0x7f1c7019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70111e40 con 0x7f1c70101de0 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70111cc0 con 0x7f1c70106f10 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70111fc0 con 0x7f1c70102790 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46936/0 (socket says 172.21.15.46:46936) 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 -- 172.21.15.46:0/2154021047 learned_addr learned my addr 172.21.15.46:0/2154021047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c74ff3640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 0x7f1c7019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c67fff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70102790 0x7f1c701988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 -- 172.21.15.46:0/2154021047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70102790 msgr2=0x7f1c701988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70102790 0x7f1c701988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 -- 172.21.15.46:0/2154021047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 msgr2=0x7f1c7019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 0x7f1c7019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 -- 172.21.15.46:0/2154021047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7019d330 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c74ff3640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 0x7f1c7019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6ffff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b4a0 tx=0x7f1c6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60014070 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7019d620 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c60010ac0 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60017c30 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.212+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c701a4f00 con 0x7f1c70101de0 2024-09-12T02:48:55.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.213+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c701021c0 con 0x7f1c70101de0 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.215+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c60017070 con 0x7f1c70101de0 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.215+0000 7f1c6dffb640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 0x7f1c5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.216+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1c600ceba0 con 0x7f1c70101de0 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.216+0000 7f1c67fff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 0x7f1c5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.216+0000 7f1c67fff640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 0x7f1c5c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005fd0 tx=0x7f1c58005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:55.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.219+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c6009b750 con 0x7f1c70101de0 2024-09-12T02:48:55.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.397+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c70109b60 con 0x7f1c70101de0 2024-09-12T02:48:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.398+0000 7f1c6dffb640 1 -- 172.21.15.46:0/2154021047 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24656 (secure 0 0 0) 0x7f1c6009b050 con 0x7f1c70101de0 2024-09-12T02:48:55.418 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:48:55.418 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:48:37.333173+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.401+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 msgr2=0x7f1c5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.401+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 0x7f1c5c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005fd0 tx=0x7f1c58005850 comp rx=0 tx=0).stop 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.401+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 msgr2=0x7f1c70198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.401+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b4a0 tx=0x7f1c6000b960 comp rx=0 tx=0).stop 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 shutdown_connections 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c70106f10 0x7f1c7019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f1c5c0a2c10 0x7f1c5c0a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c70102790 0x7f1c701988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 --2- 172.21.15.46:0/2154021047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c70101de0 0x7f1c70198380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 >> 172.21.15.46:0/2154021047 conn(0x7f1c700fd8c0 msgr2=0x7f1c700fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 shutdown_connections 2024-09-12T02:48:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:55.402+0000 7f1c76a56640 1 -- 172.21.15.46:0/2154021047 wait complete. 2024-09-12T02:48:56.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T02:48:56.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T02:48:56.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T02:48:56.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T02:48:56.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T02:48:56.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T02:48:56.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T02:48:56.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T02:48:56.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T02:48:56.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T02:48:56.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T02:48:56.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T02:48:56.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:56 smithi046 ceph-mon[30951]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:56.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:56 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2154021047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:56 smithi145 ceph-mon[34674]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:56 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2154021047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:56.704 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:56 smithi196 ceph-mon[34587]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:56 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2154021047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:48:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.061 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.061 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:57.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:48:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- 172.21.15.46:0/2823077190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 msgr2=0x7f8630107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 --2- 172.21.15.46:0/2823077190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630107b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8624009930 tx=0x7f862402a440 comp rx=0 tx=0).stop 2024-09-12T02:48:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- 172.21.15.46:0/2823077190 shutdown_connections 2024-09-12T02:48:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 --2- 172.21.15.46:0/2823077190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86301082a0 0x7f863010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 --2- 172.21.15.46:0/2823077190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630107b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 --2- 172.21.15.46:0/2823077190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8630102d30 0x7f8630103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- 172.21.15.46:0/2823077190 >> 172.21.15.46:0/2823077190 conn(0x7f86300fc7d0 msgr2=0x7f86300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- 172.21.15.46:0/2823077190 shutdown_connections 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- 172.21.15.46:0/2823077190 wait complete. 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 Processor -- start 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 -- start start 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.315+0000 7f8637af0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8630102d30 0x7f8630100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86301082a0 0x7f8630101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8630111f10 con 0x7f86301036e0 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8630111d90 con 0x7f8630102d30 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8630112090 con 0x7f86301082a0 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46944/0 (socket says 172.21.15.46:46944) 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 -- 172.21.15.46:0/3097018841 learned_addr learned my addr 172.21.15.46:0/3097018841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863588c640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8630102d30 0x7f8630100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 -- 172.21.15.46:0/3097018841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86301082a0 msgr2=0x7f8630101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863608d640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86301082a0 0x7f8630101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86301082a0 0x7f8630101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 -- 172.21.15.46:0/3097018841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8630102d30 msgr2=0x7f8630100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8630102d30 0x7f8630100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86301a68a0 con 0x7f86301036e0 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863588c640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8630102d30 0x7f8630100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f863508b640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630100ed0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8624009230 tx=0x7f8624004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862403f030 con 0x7f86301036e0 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86301a6b30 con 0x7f86301036e0 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86240035f0 con 0x7f86301036e0 2024-09-12T02:48:58.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.316+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624035070 con 0x7f86301036e0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.317+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a7010 con 0x7f86301036e0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.317+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86300628c0 con 0x7f86301036e0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.319+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8624003aa0 con 0x7f86301036e0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f862effd640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86180a2c10 0x7f86180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f863588c640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86180a2c10 0x7f86180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f86240f05a0 con 0x7f86301036e0 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f862effd640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] conn(0x7f86180adde0 0x7f86180b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8624009080 con 0x7f86180adde0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f86240f09f0 con 0x7f86301036e0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f863608d640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] conn(0x7f86180adde0 0x7f86180b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f863588c640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86180a2c10 0x7f86180a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8620007e70 tx=0x7f8620007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.320+0000 7f863608d640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] conn(0x7f86180adde0 0x7f86180b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.321+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== osd.0 v2:172.21.15.46:6802/836973982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8624009080 con 0x7f86180adde0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 --> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8630062cc0 con 0x7f86180adde0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f862effd640 1 -- 172.21.15.46:0/3097018841 <== osd.0 v2:172.21.15.46:6802/836973982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8630062cc0 con 0x7f86180adde0 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] conn(0x7f86180adde0 msgr2=0x7f86180b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] conn(0x7f86180adde0 0x7f86180b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86180a2c10 msgr2=0x7f86180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f86180a2c10 0x7f86180a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8620007e70 tx=0x7f8620007850 comp rx=0 tx=0).stop 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 msgr2=0x7f8630100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 --2- 172.21.15.46:0/3097018841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86301036e0 0x7f8630100ed0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8624009230 tx=0x7f8624004270 comp rx=0 tx=0).stop 2024-09-12T02:48:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f863608d640 1 -- 172.21.15.46:0/3097018841 reap_dead start 2024-09-12T02:48:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 shutdown_connections 2024-09-12T02:48:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 >> 172.21.15.46:0/3097018841 conn(0x7f86300fc7d0 msgr2=0x7f8630105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 shutdown_connections 2024-09-12T02:48:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:57.336+0000 7f8637af0640 1 -- 172.21.15.46:0/3097018841 wait complete. 2024-09-12T02:48:58.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:48:58 smithi046 ceph-mon[30951]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:48:58 smithi145 ceph-mon[34674]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:48:58 smithi196 ceph-mon[34587]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T02:48:58.809 INFO:teuthology.orchestra.run.smithi046.stdout:34359738453 2024-09-12T02:48:58.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 -- 172.21.15.46:0/2029794308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c106a60 msgr2=0x7f220c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 --2- 172.21.15.46:0/2029794308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c106a60 0x7f220c106e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2208009930 tx=0x7f220802a420 comp rx=0 tx=0).stop 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 -- 172.21.15.46:0/2029794308 shutdown_connections 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 --2- 172.21.15.46:0/2029794308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c101400 0x7f220c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 --2- 172.21.15.46:0/2029794308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f220c100a60 0x7f220c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 --2- 172.21.15.46:0/2029794308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c106a60 0x7f220c106e40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 -- 172.21.15.46:0/2029794308 >> 172.21.15.46:0/2029794308 conn(0x7f220c0fc780 msgr2=0x7f220c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 -- 172.21.15.46:0/2029794308 shutdown_connections 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.987+0000 7f2213cd6640 1 -- 172.21.15.46:0/2029794308 wait complete. 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 Processor -- start 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 -- start start 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f220c100a60 0x7f220c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 0x7f220c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c106a60 0x7f220c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f220c110d30 con 0x7f220c101400 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f220c110bb0 con 0x7f220c100a60 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2213cd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f220c110eb0 con 0x7f220c106a60 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2211271640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 0x7f220c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2211271640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 0x7f220c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46974/0 (socket says 172.21.15.46:46974) 2024-09-12T02:48:58.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2211271640 1 -- 172.21.15.46:0/1711001707 learned_addr learned my addr 172.21.15.46:0/1711001707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.988+0000 7f2211a72640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f220c100a60 0x7f220c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2212273640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c106a60 0x7f220c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 -- 172.21.15.46:0/1711001707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c106a60 msgr2=0x7f220c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c106a60 0x7f220c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 -- 172.21.15.46:0/1711001707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f220c100a60 msgr2=0x7f220c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f220c100a60 0x7f220c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c19c2a0 con 0x7f220c101400 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2212273640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c106a60 0x7f220c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2211271640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 0x7f220c197830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b4a0 tx=0x7f21fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc014070 con 0x7f220c101400 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21fc010ac0 con 0x7f220c101400 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c19c590 con 0x7f220c101400 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc017c30 con 0x7f220c101400 2024-09-12T02:48:58.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.989+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a3e70 con 0x7f220c101400 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.990+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f220c062980 con 0x7f220c101400 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.991+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f21fc017070 con 0x7f220c101400 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2202ffd640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f21f80a2c90 0x7f21f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f21fc0cecc0 con 0x7f220c101400 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2211a72640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f21f80a2c90 0x7f21f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2202ffd640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] conn(0x7f21f80ade60 0x7f21f80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21f80b09d0 con 0x7f21f80ade60 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.992+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f21fc0cf070 con 0x7f220c101400 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.993+0000 7f2212273640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] conn(0x7f21f80ade60 0x7f21f80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.993+0000 7f2211a72640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f21f80a2c90 0x7f21f80a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2208006010 tx=0x7f2208000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.993+0000 7f2212273640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] conn(0x7f21f80ade60 0x7f21f80b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:59.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:58.994+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== osd.6 v2:172.21.15.46:6818/2825700382 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21f80b09d0 con 0x7f21f80ade60 2024-09-12T02:48:59.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.009+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 --> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f220c062d80 con 0x7f21f80ade60 2024-09-12T02:48:59.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.009+0000 7f2202ffd640 1 -- 172.21.15.46:0/1711001707 <== osd.6 v2:172.21.15.46:6818/2825700382 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f220c062d80 con 0x7f21f80ade60 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] conn(0x7f21f80ade60 msgr2=0x7f21f80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] conn(0x7f21f80ade60 0x7f21f80b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f21f80a2c90 msgr2=0x7f21f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f21f80a2c90 0x7f21f80a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2208006010 tx=0x7f2208000bc0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 msgr2=0x7f220c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 --2- 172.21.15.46:0/1711001707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220c101400 0x7f220c197830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b4a0 tx=0x7f21fc00b960 comp rx=0 tx=0).stop 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2212273640 1 -- 172.21.15.46:0/1711001707 reap_dead start 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 shutdown_connections 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 >> 172.21.15.46:0/1711001707 conn(0x7f220c0fc780 msgr2=0x7f220c0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:59.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 shutdown_connections 2024-09-12T02:48:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.010+0000 7f2213cd6640 1 -- 172.21.15.46:0/1711001707 wait complete. 2024-09-12T02:48:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 -- 172.21.15.46:0/2120799000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4102d40 msgr2=0x7f16b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/2120799000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4102d40 0x7f16b4103120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009900 tx=0x7f16b002a480 comp rx=0 tx=0).stop 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 -- 172.21.15.46:0/2120799000 shutdown_connections 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/2120799000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16b410c170 0x7f16b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/2120799000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16b4103660 0x7f16b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/2120799000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4102d40 0x7f16b4103120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 -- 172.21.15.46:0/2120799000 >> 172.21.15.46:0/2120799000 conn(0x7f16b40fc7a0 msgr2=0x7f16b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 -- 172.21.15.46:0/2120799000 shutdown_connections 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.196+0000 7f16bbbd3640 1 -- 172.21.15.46:0/2120799000 wait complete. 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 Processor -- start 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 -- start start 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16b4102d40 0x7f16b41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 0x7f16b4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16b410c170 0x7f16b419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4111e60 con 0x7f16b4103660 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4111ce0 con 0x7f16b410c170 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16bbbd3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4111fe0 con 0x7f16b4102d40 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 0x7f16b4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 0x7f16b4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46998/0 (socket says 172.21.15.46:46998) 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 -- 172.21.15.46:0/321802691 learned_addr learned my addr 172.21.15.46:0/321802691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b996f640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16b4102d40 0x7f16b41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16ba170640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16b410c170 0x7f16b419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 -- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16b4102d40 msgr2=0x7f16b41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16b4102d40 0x7f16b41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.197+0000 7f16b916e640 1 -- 172.21.15.46:0/321802691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16b410c170 msgr2=0x7f16b419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16b916e640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16b410c170 0x7f16b419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16b916e640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b419d380 con 0x7f16b4103660 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16b916e640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 0x7f16b4198910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d740 tx=0x7f16a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8017070 con 0x7f16b4103660 2024-09-12T02:48:59.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b419d670 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a800de50 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800f620 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.198+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a4f50 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.199+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16b4062980 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.200+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16a8009c30 con 0x7f16b4103660 2024-09-12T02:48:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16a6ffd640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16a00a2c10 0x7f16a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16b996f640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16a00a2c10 0x7f16a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f16a80cf350 con 0x7f16b4103660 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16a6ffd640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] conn(0x7f16a00adde0 0x7f16a00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16a8010a00 con 0x7f16a00adde0 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.201+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f16a80cf830 con 0x7f16b4103660 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.202+0000 7f16b996f640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16a00a2c10 0x7f16a00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031440 tx=0x7f16b0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.202+0000 7f16ba170640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] conn(0x7f16a00adde0 0x7f16a00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.202+0000 7f16ba170640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] conn(0x7f16a00adde0 0x7f16a00b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.203+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== osd.5 v2:172.21.15.196:6808/859658761 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16a8010a00 con 0x7f16a00adde0 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.217+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 --> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16b4062d80 con 0x7f16a00adde0 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.217+0000 7f16a6ffd640 1 -- 172.21.15.46:0/321802691 <== osd.5 v2:172.21.15.196:6808/859658761 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16b4062d80 con 0x7f16a00adde0 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] conn(0x7f16a00adde0 msgr2=0x7f16a00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] conn(0x7f16a00adde0 0x7f16a00b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16a00a2c10 msgr2=0x7f16a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f16a00a2c10 0x7f16a00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031440 tx=0x7f16b0005b50 comp rx=0 tx=0).stop 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 msgr2=0x7f16b4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 --2- 172.21.15.46:0/321802691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b4103660 0x7f16b4198910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d740 tx=0x7f16a800dc00 comp rx=0 tx=0).stop 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16ba170640 1 -- 172.21.15.46:0/321802691 reap_dead start 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 shutdown_connections 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 >> 172.21.15.46:0/321802691 conn(0x7f16b40fc7a0 msgr2=0x7f16b40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:59.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 shutdown_connections 2024-09-12T02:48:59.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:48:59.218+0000 7f16bbbd3640 1 -- 172.21.15.46:0/321802691 wait complete. 2024-09-12T02:48:59.858 INFO:teuthology.orchestra.run.smithi046.stdout:163208757308 2024-09-12T02:48:59.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T02:49:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.011+0000 7ff7651f7640 1 -- 172.21.15.46:0/3286207452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff760102d30 msgr2=0x7ff760103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.011+0000 7ff7651f7640 1 --2- 172.21.15.46:0/3286207452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff760102d30 0x7ff760103110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff750009930 tx=0x7ff75002a420 comp rx=0 tx=0).stop 2024-09-12T02:49:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 -- 172.21.15.46:0/3286207452 shutdown_connections 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 --2- 172.21.15.46:0/3286207452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7601082a0 0x7ff76010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 --2- 172.21.15.46:0/3286207452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7601036e0 0x7ff760107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 --2- 172.21.15.46:0/3286207452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff760102d30 0x7ff760103110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 -- 172.21.15.46:0/3286207452 >> 172.21.15.46:0/3286207452 conn(0x7ff7600fc7d0 msgr2=0x7ff7600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 -- 172.21.15.46:0/3286207452 shutdown_connections 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 -- 172.21.15.46:0/3286207452 wait complete. 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.012+0000 7ff7651f7640 1 Processor -- start 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 -- start start 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff760102d30 0x7ff760100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 0x7ff760100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7601082a0 0x7ff760101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff760111f50 con 0x7ff7601036e0 2024-09-12T02:49:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff760111dd0 con 0x7ff760102d30 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff7651f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601120d0 con 0x7ff7601082a0 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 0x7ff760100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 0x7ff760100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47028/0 (socket says 172.21.15.46:47028) 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 -- 172.21.15.46:0/2269822674 learned_addr learned my addr 172.21.15.46:0/2269822674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75ed9d640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff760102d30 0x7ff760100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75f59e640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7601082a0 0x7ff760101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 -- 172.21.15.46:0/2269822674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7601082a0 msgr2=0x7ff760101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7601082a0 0x7ff760101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 -- 172.21.15.46:0/2269822674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff760102d30 msgr2=0x7ff760100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff760102d30 0x7ff760100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.013+0000 7ff75e59c640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7601a69b0 con 0x7ff7601036e0 2024-09-12T02:49:00.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff75e59c640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 0x7ff760100ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9c0 tx=0x7ff75400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75400ccc0 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7601a6ca0 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7540054d0 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754010430 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.014+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601a7190 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.015+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7600628c0 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.016+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff754005000 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff747fff640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff7480a2c10 0x7ff7480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff75ed9d640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff7480a2c10 0x7ff7480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff7540d0040 con 0x7ff7601036e0 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff747fff640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] conn(0x7ff7480adde0 0x7ff7480b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff754020e50 con 0x7ff7480adde0 2024-09-12T02:49:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff7540d0490 con 0x7ff7601036e0 2024-09-12T02:49:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff75f59e640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] conn(0x7ff7480adde0 0x7ff7480b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.017+0000 7ff75ed9d640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff7480a2c10 0x7ff7480a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.018+0000 7ff75f59e640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] conn(0x7ff7480adde0 0x7ff7480b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.018+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== osd.9 v2:172.21.15.46:6826/182095023 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff754020e50 con 0x7ff7480adde0 2024-09-12T02:49:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.032+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 --> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff760062d40 con 0x7ff7480adde0 2024-09-12T02:49:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.032+0000 7ff747fff640 1 -- 172.21.15.46:0/2269822674 <== osd.9 v2:172.21.15.46:6826/182095023 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff760062d40 con 0x7ff7480adde0 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] conn(0x7ff7480adde0 msgr2=0x7ff7480b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] conn(0x7ff7480adde0 0x7ff7480b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff7480a2c10 msgr2=0x7ff7480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff7480a2c10 0x7ff7480a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750000bc0 comp rx=0 tx=0).stop 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 msgr2=0x7ff760100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 --2- 172.21.15.46:0/2269822674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7601036e0 0x7ff760100ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9c0 tx=0x7ff75400ee80 comp rx=0 tx=0).stop 2024-09-12T02:49:00.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff75f59e640 1 -- 172.21.15.46:0/2269822674 reap_dead start 2024-09-12T02:49:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 shutdown_connections 2024-09-12T02:49:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 >> 172.21.15.46:0/2269822674 conn(0x7ff7600fc7d0 msgr2=0x7ff760105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 shutdown_connections 2024-09-12T02:49:00.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:00.033+0000 7ff7651f7640 1 -- 172.21.15.46:0/2269822674 wait complete. 2024-09-12T02:49:00.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:00 smithi046 ceph-mon[30951]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:49:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:00 smithi145 ceph-mon[34674]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:49:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:00 smithi196 ceph-mon[34587]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T02:49:01.312 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:01.313 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:01.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:01 smithi046 ceph-mon[30951]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:01.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:01 smithi145 ceph-mon[34674]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:01 smithi196 ceph-mon[34587]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:01.753 INFO:teuthology.orchestra.run.smithi046.stdout:193273528374 2024-09-12T02:49:01.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T02:49:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 -- 172.21.15.46:0/66367464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0101420 msgr2=0x7f2cf010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/66367464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0101420 0x7f2cf010d950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009cd0 tx=0x7f2ce802a8f0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 -- 172.21.15.46:0/66367464 shutdown_connections 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/66367464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0101420 0x7f2cf010d950 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/66367464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cf0100a80 0x7f2cf0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/66367464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0106a80 0x7f2cf0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 -- 172.21.15.46:0/66367464 >> 172.21.15.46:0/66367464 conn(0x7f2cf00fc7a0 msgr2=0x7f2cf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 -- 172.21.15.46:0/66367464 shutdown_connections 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.199+0000 7f2cf4cae640 1 -- 172.21.15.46:0/66367464 wait complete. 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 Processor -- start 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 -- start start 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cf0100a80 0x7f2cf01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101420 0x7f2cf01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 0x7f2cf019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:02.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110d30 con 0x7f2cf0106a80 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110bb0 con 0x7f2cf0101420 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cf4cae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110eb0 con 0x7f2cf0100a80 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 0x7f2cf019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 0x7f2cf019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47036/0 (socket says 172.21.15.46:47036) 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2ceed9d640 1 -- 172.21.15.46:0/824198051 learned_addr learned my addr 172.21.15.46:0/824198051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.200+0000 7f2cedd9b640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101420 0x7f2cf01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cee59c640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cf0100a80 0x7f2cf01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 -- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cf0100a80 msgr2=0x7f2cf01971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cf0100a80 0x7f2cf01971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 -- 172.21.15.46:0/824198051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101420 msgr2=0x7f2cf01976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101420 0x7f2cf01976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf019c160 con 0x7f2cf0106a80 2024-09-12T02:49:02.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2ceed9d640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 0x7f2cf019ba80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005cc0 tx=0x7f2ce8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8049070 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf019c3f0 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce80044f0 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce803abb0 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.201+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf006a790 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.202+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2cf006ad40 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.203+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2ce804c070 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2cd80a2c10 0x7f2cd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cee59c640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2cd80a2c10 0x7f2cd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2ce80f19b0 con 0x7f2cf0106a80 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] conn(0x7f2cd80adde0 0x7f2cd80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ce803ce50 con 0x7f2cd80adde0 2024-09-12T02:49:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2ce80f1e00 con 0x7f2cf0106a80 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.204+0000 7f2cee59c640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2cd80a2c10 0x7f2cd80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0005d90 tx=0x7f2ce0005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.205+0000 7f2cedd9b640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] conn(0x7f2cd80adde0 0x7f2cd80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.205+0000 7f2cedd9b640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] conn(0x7f2cd80adde0 0x7f2cd80b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.206+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== osd.11 v2:172.21.15.196:6824/3261621208 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ce803ce50 con 0x7f2cd80adde0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.219+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 --> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2cf006b1c0 con 0x7f2cd80adde0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cdf7fe640 1 -- 172.21.15.46:0/824198051 <== osd.11 v2:172.21.15.196:6824/3261621208 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2cf006b1c0 con 0x7f2cd80adde0 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] conn(0x7f2cd80adde0 msgr2=0x7f2cd80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] conn(0x7f2cd80adde0 0x7f2cd80b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2cd80a2c10 msgr2=0x7f2cd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f2cd80a2c10 0x7f2cd80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0005d90 tx=0x7f2ce0005d20 comp rx=0 tx=0).stop 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 msgr2=0x7f2cf019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2cf4cae640 1 --2- 172.21.15.46:0/824198051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cf0106a80 0x7f2cf019ba80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005cc0 tx=0x7f2ce8003ba0 comp rx=0 tx=0).stop 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.220+0000 7f2ceed9d640 1 -- 172.21.15.46:0/824198051 reap_dead start 2024-09-12T02:49:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.221+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 shutdown_connections 2024-09-12T02:49:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.221+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 >> 172.21.15.46:0/824198051 conn(0x7f2cf00fc7a0 msgr2=0x7f2cf00fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.221+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 shutdown_connections 2024-09-12T02:49:02.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:02.221+0000 7f2cf4cae640 1 -- 172.21.15.46:0/824198051 wait complete. 2024-09-12T02:49:03.487 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:03 smithi046 ceph-mon[30951]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/1309408399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68101420 msgr2=0x7fcc6810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/1309408399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68101420 0x7fcc6810d950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009cd0 tx=0x7fcc6002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/1309408399 shutdown_connections 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/1309408399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68101420 0x7fcc6810d950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/1309408399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc68100a80 0x7fcc68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/1309408399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 0x7fcc68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/1309408399 >> 172.21.15.46:0/1309408399 conn(0x7fcc680fc7a0 msgr2=0x7fcc680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/1309408399 shutdown_connections 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.485+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/1309408399 wait complete. 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 Processor -- start 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 -- start start 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 0x7fcc681a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc68101420 0x7fcc681a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 0x7fcc6819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68110d30 con 0x7fcc68100a80 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68110bb0 con 0x7fcc68106a80 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6cfc2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68110eb0 con 0x7fcc68101420 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.486+0000 7fcc6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 0x7fcc681a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 0x7fcc681a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47054/0 (socket says 172.21.15.46:47054) 2024-09-12T02:49:03.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 -- 172.21.15.46:0/3126032471 learned_addr learned my addr 172.21.15.46:0/3126032471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc5dd9b640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc68101420 0x7fcc681a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc66d9d640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 0x7fcc6819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 -- 172.21.15.46:0/3126032471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc68101420 msgr2=0x7fcc681a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc68101420 0x7fcc681a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 -- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 msgr2=0x7fcc6819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 0x7fcc6819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6819f240 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc66d9d640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc68106a80 0x7fcc6819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6659c640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 0x7fcc681a4630 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c920 tx=0x7fcc5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58007c70 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6819f530 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc58007e10 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58018820 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.487+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc681ab540 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.488+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc680fd6f0 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.490+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcc580189c0 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.490+0000 7fcc5ffff640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc540a2c10 0x7fcc540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcc580cfa80 con 0x7fcc68100a80 2024-09-12T02:49:03.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5dd9b640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc540a2c10 0x7fcc540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5ffff640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] conn(0x7fcc540adde0 0x7fcc540b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc58015e70 con 0x7fcc540adde0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcc580cfed0 con 0x7fcc68100a80 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc5dd9b640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc540a2c10 0x7fcc540a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc6810de20 tx=0x7fcc5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc66d9d640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] conn(0x7fcc540adde0 0x7fcc540b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.491+0000 7fcc66d9d640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] conn(0x7fcc540adde0 0x7fcc540b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.492+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== osd.10 v2:172.21.15.145:6824/140848472 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc58015e70 con 0x7fcc540adde0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.506+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 --> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc681021c0 con 0x7fcc540adde0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.506+0000 7fcc5ffff640 1 -- 172.21.15.46:0/3126032471 <== osd.10 v2:172.21.15.145:6824/140848472 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc681021c0 con 0x7fcc540adde0 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] conn(0x7fcc540adde0 msgr2=0x7fcc540b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] conn(0x7fcc540adde0 0x7fcc540b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc540a2c10 msgr2=0x7fcc540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcc540a2c10 0x7fcc540a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc6810de20 tx=0x7fcc5000b390 comp rx=0 tx=0).stop 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 msgr2=0x7fcc681a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 --2- 172.21.15.46:0/3126032471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc68100a80 0x7fcc681a4630 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c920 tx=0x7fcc5800cde0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc66d9d640 1 -- 172.21.15.46:0/3126032471 reap_dead start 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 shutdown_connections 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 >> 172.21.15.46:0/3126032471 conn(0x7fcc680fc7a0 msgr2=0x7fcc680fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 shutdown_connections 2024-09-12T02:49:03.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.507+0000 7fcc6cfc2640 1 -- 172.21.15.46:0/3126032471 wait complete. 2024-09-12T02:49:03.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:03 smithi145 ceph-mon[34674]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:03 smithi196 ceph-mon[34587]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- 172.21.15.46:0/3030635552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 msgr2=0x7f88241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 --2- 172.21.15.46:0/3030635552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241021c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8818009900 tx=0x7f881802a3f0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- 172.21.15.46:0/3030635552 shutdown_connections 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 --2- 172.21.15.46:0/3030635552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824106f10 0x7f882410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 --2- 172.21.15.46:0/3030635552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8824102790 0x7f88241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 --2- 172.21.15.46:0/3030635552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241021c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- 172.21.15.46:0/3030635552 >> 172.21.15.46:0/3030635552 conn(0x7f88240fd8c0 msgr2=0x7f88240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- 172.21.15.46:0/3030635552 shutdown_connections 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- 172.21.15.46:0/3030635552 wait complete. 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 Processor -- start 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.923+0000 7f882c736640 1 -- start start 2024-09-12T02:49:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824102790 0x7f88241987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8824106f10 0x7f882419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8824111de0 con 0x7f8824101de0 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8824111c60 con 0x7f8824102790 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882c736640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8824111f60 con 0x7f8824106f10 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47086/0 (socket says 172.21.15.46:47086) 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 -- 172.21.15.46:0/2986296236 learned_addr learned my addr 172.21.15.46:0/2986296236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882acd3640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8824106f10 0x7f882419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f8829cd1640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824102790 0x7f88241987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 -- 172.21.15.46:0/2986296236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8824106f10 msgr2=0x7f882419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8824106f10 0x7f882419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 -- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824102790 msgr2=0x7f88241987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824102790 0x7f88241987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882419d260 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f8829cd1640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8824102790 0x7f88241987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.924+0000 7f882a4d2640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241982b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8818005940 tx=0x7f8818004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881803f030 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882419d4f0 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8818003aa0 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818035070 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a4e30 con 0x7f8824101de0 2024-09-12T02:49:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.925+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8824062980 con 0x7f8824101de0 2024-09-12T02:49:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.927+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8818042030 con 0x7f8824101de0 2024-09-12T02:49:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f88177fe640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88100a2c10 0x7f88100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f8829cd1640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88100a2c10 0x7f88100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f88180f0490 con 0x7f8824101de0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f88177fe640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] conn(0x7f88100adde0 0x7f88100b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88100b0950 con 0x7f88100adde0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f88180f0840 con 0x7f8824101de0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f8829cd1640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88100a2c10 0x7f88100a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88240fdce0 tx=0x7f8820005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.928+0000 7f882acd3640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] conn(0x7f88100adde0 0x7f88100b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.929+0000 7f882acd3640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] conn(0x7f88100adde0 0x7f88100b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:03.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.930+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== osd.1 v2:172.21.15.145:6800/1956814217 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88100b0950 con 0x7f88100adde0 2024-09-12T02:49:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.944+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 --> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8824062e00 con 0x7f88100adde0 2024-09-12T02:49:03.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f88177fe640 1 -- 172.21.15.46:0/2986296236 <== osd.1 v2:172.21.15.145:6800/1956814217 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8824062e00 con 0x7f88100adde0 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] conn(0x7f88100adde0 msgr2=0x7f88100b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] conn(0x7f88100adde0 0x7f88100b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88100a2c10 msgr2=0x7f88100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f88100a2c10 0x7f88100a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88240fdce0 tx=0x7f8820005d90 comp rx=0 tx=0).stop 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 msgr2=0x7f88241982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 --2- 172.21.15.46:0/2986296236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8824101de0 0x7f88241982b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8818005940 tx=0x7f8818004270 comp rx=0 tx=0).stop 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882acd3640 1 -- 172.21.15.46:0/2986296236 reap_dead start 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 shutdown_connections 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 >> 172.21.15.46:0/2986296236 conn(0x7f88240fd8c0 msgr2=0x7f88240fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 shutdown_connections 2024-09-12T02:49:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:03.945+0000 7f882c736640 1 -- 172.21.15.46:0/2986296236 wait complete. 2024-09-12T02:49:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.124+0000 7f227caef640 1 -- 172.21.15.46:0/3451707542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278102790 msgr2=0x7f22781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.124+0000 7f227caef640 1 --2- 172.21.15.46:0/3451707542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278102790 0x7f22781067e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2260009900 tx=0x7f226002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 -- 172.21.15.46:0/3451707542 shutdown_connections 2024-09-12T02:49:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 --2- 172.21.15.46:0/3451707542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278106f10 0x7f227810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 --2- 172.21.15.46:0/3451707542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278102790 0x7f22781067e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 --2- 172.21.15.46:0/3451707542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 0x7f22781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 -- 172.21.15.46:0/3451707542 >> 172.21.15.46:0/3451707542 conn(0x7f22780fd8c0 msgr2=0x7f22780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 -- 172.21.15.46:0/3451707542 shutdown_connections 2024-09-12T02:49:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 -- 172.21.15.46:0/3451707542 wait complete. 2024-09-12T02:49:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.125+0000 7f227caef640 1 Processor -- start 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 -- start start 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 0x7f2278198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278102790 0x7f22781988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 0x7f227819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111e20 con 0x7f2278106f10 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111ca0 con 0x7f2278102790 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227caef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111fa0 con 0x7f2278101de0 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 0x7f227819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 0x7f227819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47110/0 (socket says 172.21.15.46:47110) 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 -- 172.21.15.46:0/3534005286 learned_addr learned my addr 172.21.15.46:0/3534005286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227659c640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 0x7f2278198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2275d9b640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278102790 0x7f22781988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 -- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 msgr2=0x7f2278198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 0x7f2278198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 -- 172.21.15.46:0/3534005286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278102790 msgr2=0x7f22781988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278102790 0x7f22781988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227819d340 con 0x7f2278106f10 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f227659c640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2278101de0 0x7f2278198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2275d9b640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278102790 0x7f22781988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.126+0000 7f2276d9d640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 0x7f227819cc60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f227000eb10 tx=0x7f227000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.127+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270004b20 con 0x7f2278106f10 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.127+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2270004cc0 con 0x7f2278106f10 2024-09-12T02:49:04.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.127+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227819d630 con 0x7f2278106f10 2024-09-12T02:49:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.127+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270010690 con 0x7f2278106f10 2024-09-12T02:49:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.127+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781a4f10 con 0x7f2278106f10 2024-09-12T02:49:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.128+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2278062980 con 0x7f2278106f10 2024-09-12T02:49:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.129+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2270005000 con 0x7f2278106f10 2024-09-12T02:49:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f226f7fe640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f22640a2c10 0x7f22640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f227659c640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f22640a2c10 0x7f22640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f22700cfff0 con 0x7f2278106f10 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f226f7fe640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] conn(0x7f22640adde0 0x7f22640b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22640b0950 con 0x7f22640adde0 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f22700d0400 con 0x7f2278106f10 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.130+0000 7f227659c640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f22640a2c10 0x7f22640a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.131+0000 7f2275d9b640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] conn(0x7f22640adde0 0x7f22640b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.131+0000 7f2275d9b640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] conn(0x7f22640adde0 0x7f22640b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.132+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== osd.8 v2:172.21.15.196:6816/724845953 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22640b0950 con 0x7f22640adde0 2024-09-12T02:49:04.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.146+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 --> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2278062e00 con 0x7f22640adde0 2024-09-12T02:49:04.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.146+0000 7f226f7fe640 1 -- 172.21.15.46:0/3534005286 <== osd.8 v2:172.21.15.196:6816/724845953 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2278062e00 con 0x7f22640adde0 2024-09-12T02:49:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] conn(0x7f22640adde0 msgr2=0x7f22640b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] conn(0x7f22640adde0 0x7f22640b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f22640a2c10 msgr2=0x7f22640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f22640a2c10 0x7f22640a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268005850 comp rx=0 tx=0).stop 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 msgr2=0x7f227819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 --2- 172.21.15.46:0/3534005286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2278106f10 0x7f227819cc60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f227000eb10 tx=0x7f227000c2d0 comp rx=0 tx=0).stop 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f2276d9d640 1 -- 172.21.15.46:0/3534005286 reap_dead start 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 shutdown_connections 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 >> 172.21.15.46:0/3534005286 conn(0x7f22780fd8c0 msgr2=0x7f22780fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 shutdown_connections 2024-09-12T02:49:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:04.147+0000 7f227caef640 1 -- 172.21.15.46:0/3534005286 wait complete. 2024-09-12T02:49:04.958 INFO:teuthology.orchestra.run.smithi046.stdout:296352743459 2024-09-12T02:49:04.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T02:49:05.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- 172.21.15.46:0/2182744202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920108300 msgr2=0x7f592010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 --2- 172.21.15.46:0/2182744202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920108300 0x7f592010ec70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5918009930 tx=0x7f591802a420 comp rx=0 tx=0).stop 2024-09-12T02:49:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- 172.21.15.46:0/2182744202 shutdown_connections 2024-09-12T02:49:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 --2- 172.21.15.46:0/2182744202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920108300 0x7f592010ec70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 --2- 172.21.15.46:0/2182744202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920103780 0x7f5920107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 --2- 172.21.15.46:0/2182744202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920102dd0 0x7f59201031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- 172.21.15.46:0/2182744202 >> 172.21.15.46:0/2182744202 conn(0x7f59200fc7d0 msgr2=0x7f59200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- 172.21.15.46:0/2182744202 shutdown_connections 2024-09-12T02:49:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- 172.21.15.46:0/2182744202 wait complete. 2024-09-12T02:49:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 Processor -- start 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.134+0000 7f59280e6640 1 -- start start 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 0x7f59201984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920103780 0x7f5920198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920108300 0x7f592019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5920111fd0 con 0x7f5920102dd0 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5920111e50 con 0x7f5920103780 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5920112150 con 0x7f5920108300 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 0x7f59201984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 0x7f59201984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49570/0 (socket says 172.21.15.46:49570) 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 -- 172.21.15.46:0/951498854 learned_addr learned my addr 172.21.15.46:0/951498854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5926683640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920108300 0x7f592019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925681640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920103780 0x7f5920198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 -- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920108300 msgr2=0x7f592019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920108300 0x7f592019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 -- 172.21.15.46:0/951498854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920103780 msgr2=0x7f5920198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920103780 0x7f5920198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592019d4a0 con 0x7f5920102dd0 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5926683640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920108300 0x7f592019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925681640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5920103780 0x7f5920198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5925e82640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 0x7f59201984f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f591400eab0 tx=0x7f591400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400ce40 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5914011920 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.135+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592019d790 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.136+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914010a00 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.136+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a5070 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.136+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5920062980 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.138+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5914011ad0 con 0x7f5920102dd0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.138+0000 7f5912ffd640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f590c0a2c90 0x7f590c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5925681640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f590c0a2c90 0x7f590c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f59140cfbb0 con 0x7f5920102dd0 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5912ffd640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] conn(0x7f590c0ade60 0x7f590c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5914014ec0 con 0x7f590c0ade60 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f59140d0000 con 0x7f5920102dd0 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5925681640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f590c0a2c90 0x7f590c0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f591c005fd0 tx=0x7f591c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.139+0000 7f5926683640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] conn(0x7f590c0ade60 0x7f590c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.140+0000 7f5926683640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] conn(0x7f590c0ade60 0x7f590c0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.141+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== osd.2 v2:172.21.15.196:6800/3433225191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5914014ec0 con 0x7f590c0ade60 2024-09-12T02:49:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.155+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 --> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5920062d80 con 0x7f590c0ade60 2024-09-12T02:49:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f5912ffd640 1 -- 172.21.15.46:0/951498854 <== osd.2 v2:172.21.15.196:6800/3433225191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5920062d80 con 0x7f590c0ade60 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] conn(0x7f590c0ade60 msgr2=0x7f590c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] conn(0x7f590c0ade60 0x7f590c0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f590c0a2c90 msgr2=0x7f590c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f590c0a2c90 0x7f590c0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f591c005fd0 tx=0x7f591c0094d0 comp rx=0 tx=0).stop 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 msgr2=0x7f59201984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 --2- 172.21.15.46:0/951498854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5920102dd0 0x7f59201984f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f591400eab0 tx=0x7f591400ef70 comp rx=0 tx=0).stop 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f5926683640 1 -- 172.21.15.46:0/951498854 reap_dead start 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 shutdown_connections 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 >> 172.21.15.46:0/951498854 conn(0x7f59200fc7d0 msgr2=0x7f5920105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 shutdown_connections 2024-09-12T02:49:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:05.156+0000 7f59280e6640 1 -- 172.21.15.46:0/951498854 wait complete. 2024-09-12T02:49:05.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:05 smithi046 ceph-mon[30951]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:49:05.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:05 smithi145 ceph-mon[34674]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:49:05.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:05 smithi196 ceph-mon[34587]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T02:49:06.173 INFO:teuthology.orchestra.run.smithi046.stdout:55834574930 2024-09-12T02:49:06.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.364+0000 7f4728164640 1 -- 172.21.15.46:0/3806703361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720102790 msgr2=0x7f47201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.364+0000 7f4728164640 1 --2- 172.21.15.46:0/3806703361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720102790 0x7f47201067e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4710009900 tx=0x7f471002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 -- 172.21.15.46:0/3806703361 shutdown_connections 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 --2- 172.21.15.46:0/3806703361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4720106f10 0x7f472010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 --2- 172.21.15.46:0/3806703361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720102790 0x7f47201067e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 --2- 172.21.15.46:0/3806703361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4720101de0 0x7f47201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 -- 172.21.15.46:0/3806703361 >> 172.21.15.46:0/3806703361 conn(0x7f47200fd8c0 msgr2=0x7f47200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 -- 172.21.15.46:0/3806703361 shutdown_connections 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 -- 172.21.15.46:0/3806703361 wait complete. 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 Processor -- start 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.365+0000 7f4728164640 1 -- start start 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4720101de0 0x7f47200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4720102790 0x7f4720074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 0x7f4720074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111de0 con 0x7f4720106f10 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111c60 con 0x7f4720102790 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4728164640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111f60 con 0x7f4720101de0 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 0x7f4720074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 0x7f4720074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49592/0 (socket says 172.21.15.46:49592) 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 -- 172.21.15.46:0/548125988 learned_addr learned my addr 172.21.15.46:0/548125988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f47256ff640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4720102790 0x7f4720074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4725f00640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4720101de0 0x7f47200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 -- 172.21.15.46:0/548125988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4720101de0 msgr2=0x7f47200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4720101de0 0x7f47200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 -- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4720102790 msgr2=0x7f4720074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4720102790 0x7f4720074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47201a6880 con 0x7f4720106f10 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4726701640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 0x7f4720074f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f471800eb10 tx=0x7f471800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.366+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718004b20 con 0x7f4720106f10 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.367+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4718004cc0 con 0x7f4720106f10 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.367+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47201a6b70 con 0x7f4720106f10 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.367+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718010690 con 0x7f4720106f10 2024-09-12T02:49:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.367+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201a7030 con 0x7f4720106f10 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.368+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47200628c0 con 0x7f4720106f10 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.369+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4718005020 con 0x7f4720106f10 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4716ffd640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f470c0a2c10 0x7f470c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f47180cfeb0 con 0x7f4720106f10 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4725f00640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f470c0a2c10 0x7f470c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4716ffd640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] conn(0x7f470c0add60 0x7f470c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47180047f0 con 0x7f470c0add60 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f47180d0260 con 0x7f4720106f10 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f47256ff640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] conn(0x7f470c0add60 0x7f470c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.370+0000 7f4725f00640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f470c0a2c10 0x7f470c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bbe0 tx=0x7f471c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.371+0000 7f47256ff640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] conn(0x7f470c0add60 0x7f470c0b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.372+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== osd.7 v2:172.21.15.145:6816/1917623570 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47180047f0 con 0x7f470c0add60 2024-09-12T02:49:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.387+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 --> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4720062d40 con 0x7f470c0add60 2024-09-12T02:49:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4716ffd640 1 -- 172.21.15.46:0/548125988 <== osd.7 v2:172.21.15.145:6816/1917623570 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4720062d40 con 0x7f470c0add60 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] conn(0x7f470c0add60 msgr2=0x7f470c0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] conn(0x7f470c0add60 0x7f470c0b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f470c0a2c10 msgr2=0x7f470c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f470c0a2c10 0x7f470c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bbe0 tx=0x7f471c00b390 comp rx=0 tx=0).stop 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 msgr2=0x7f4720074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4728164640 1 --2- 172.21.15.46:0/548125988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4720106f10 0x7f4720074f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f471800eb10 tx=0x7f471800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.388+0000 7f4726701640 1 -- 172.21.15.46:0/548125988 reap_dead start 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.389+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 shutdown_connections 2024-09-12T02:49:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.389+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 >> 172.21.15.46:0/548125988 conn(0x7f47200fd8c0 msgr2=0x7f47200fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:06.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.389+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 shutdown_connections 2024-09-12T02:49:06.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:06.389+0000 7f4728164640 1 -- 172.21.15.46:0/548125988 wait complete. 2024-09-12T02:49:07.072 INFO:teuthology.orchestra.run.smithi046.stdout:326417514526 2024-09-12T02:49:07.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T02:49:07.518 INFO:teuthology.orchestra.run.smithi046.stdout:77309411406 2024-09-12T02:49:07.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T02:49:07.602 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:07 smithi046 ceph-mon[30951]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T02:49:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:07 smithi145 ceph-mon[34674]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T02:49:07.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:07 smithi196 ceph-mon[34587]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T02:49:08.164 INFO:teuthology.orchestra.run.smithi046.stdout:214748364851 2024-09-12T02:49:08.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T02:49:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 -- 172.21.15.46:0/507698296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d810c190 msgr2=0x7fc2d810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/507698296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d810c190 0x7fc2d810e600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009ca0 tx=0x7fc2d002a950 comp rx=0 tx=0).stop 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 -- 172.21.15.46:0/507698296 shutdown_connections 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/507698296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d810c190 0x7fc2d810e600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/507698296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d8103710 0x7fc2d810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/507698296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d8102df0 0x7fc2d81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.356+0000 7fc2dcc38640 1 -- 172.21.15.46:0/507698296 >> 172.21.15.46:0/507698296 conn(0x7fc2d80fc790 msgr2=0x7fc2d80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.357+0000 7fc2dcc38640 1 -- 172.21.15.46:0/507698296 shutdown_connections 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.357+0000 7fc2dcc38640 1 -- 172.21.15.46:0/507698296 wait complete. 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.357+0000 7fc2dcc38640 1 Processor -- start 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.357+0000 7fc2dcc38640 1 -- start start 2024-09-12T02:49:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 0x7fc2d8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d8103710 0x7fc2d8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d810c190 0x7fc2d819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8111e10 con 0x7fc2d8102df0 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8111c90 con 0x7fc2d8103710 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8111f90 con 0x7fc2d810c190 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 0x7fc2d8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 0x7fc2d8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49618/0 (socket says 172.21.15.46:49618) 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 -- 172.21.15.46:0/4232467218 learned_addr learned my addr 172.21.15.46:0/4232467218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d5d9b640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d8103710 0x7fc2d8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d6d9d640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d810c190 0x7fc2d819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 -- 172.21.15.46:0/4232467218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d810c190 msgr2=0x7fc2d819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d810c190 0x7fc2d819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 -- 172.21.15.46:0/4232467218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d8103710 msgr2=0x7fc2d8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2d8103710 0x7fc2d8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d819d2d0 con 0x7fc2d8102df0 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d6d9d640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2d810c190 0x7fc2d819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2d659c640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 0x7fc2d8198320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800d770 tx=0x7fc2c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8017070 con 0x7fc2d8102df0 2024-09-12T02:49:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d819d5c0 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2c80054d0 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.358+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004cf0 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.359+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a4ea0 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.359+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2d8062980 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.361+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc2c8004e90 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.361+0000 7fc2cf7fe640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2c40a2c10 0x7fc2c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2d5d9b640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2c40a2c10 0x7fc2c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc2c80cf310 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2cf7fe640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] conn(0x7fc2c40adde0 0x7fc2c40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2c8005320 con 0x7fc2c40adde0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc2c80cf7f0 con 0x7fc2d8102df0 2024-09-12T02:49:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2d5d9b640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2c40a2c10 0x7fc2c40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80fee00 tx=0x7fc2c000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2d6d9d640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] conn(0x7fc2c40adde0 0x7fc2c40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.362+0000 7fc2d6d9d640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] conn(0x7fc2c40adde0 0x7fc2c40b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.363+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== osd.3 v2:172.21.15.46:6810/3782480680 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2c8005320 con 0x7fc2c40adde0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 --> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2d8062e00 con 0x7fc2c40adde0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2cf7fe640 1 -- 172.21.15.46:0/4232467218 <== osd.3 v2:172.21.15.46:6810/3782480680 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc2d8062e00 con 0x7fc2c40adde0 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] conn(0x7fc2c40adde0 msgr2=0x7fc2c40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] conn(0x7fc2c40adde0 0x7fc2c40b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2c40a2c10 msgr2=0x7fc2c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2c40a2c10 0x7fc2c40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80fee00 tx=0x7fc2c000b390 comp rx=0 tx=0).stop 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 msgr2=0x7fc2d8198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2dcc38640 1 --2- 172.21.15.46:0/4232467218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2d8102df0 0x7fc2d8198320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800d770 tx=0x7fc2c800dc30 comp rx=0 tx=0).stop 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.377+0000 7fc2d6d9d640 1 -- 172.21.15.46:0/4232467218 reap_dead start 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.378+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 shutdown_connections 2024-09-12T02:49:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.378+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 >> 172.21.15.46:0/4232467218 conn(0x7fc2d80fc790 msgr2=0x7fc2d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.378+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 shutdown_connections 2024-09-12T02:49:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.378+0000 7fc2dcc38640 1 -- 172.21.15.46:0/4232467218 wait complete. 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 -- 172.21.15.46:0/2653557004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 msgr2=0x7f44501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 --2- 172.21.15.46:0/2653557004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f44501021c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4440009900 tx=0x7f444002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 -- 172.21.15.46:0/2653557004 shutdown_connections 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 --2- 172.21.15.46:0/2653557004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450106f10 0x7f445010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 --2- 172.21.15.46:0/2653557004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450102790 0x7f44501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 --2- 172.21.15.46:0/2653557004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f44501021c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 -- 172.21.15.46:0/2653557004 >> 172.21.15.46:0/2653557004 conn(0x7f44500fd8c0 msgr2=0x7f44500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 -- 172.21.15.46:0/2653557004 shutdown_connections 2024-09-12T02:49:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.873+0000 7f445674d640 1 -- 172.21.15.46:0/2653557004 wait complete. 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.874+0000 7f445674d640 1 Processor -- start 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.874+0000 7f445674d640 1 -- start start 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f4450198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450102790 0x7f4450198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450106f10 0x7f445019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111e20 con 0x7f4450101de0 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111ca0 con 0x7f4450102790 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f445674d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111fa0 con 0x7f4450106f10 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f4450198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f4450198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49634/0 (socket says 172.21.15.46:49634) 2024-09-12T02:49:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 -- 172.21.15.46:0/1885600484 learned_addr learned my addr 172.21.15.46:0/1885600484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f4447fff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450102790 0x7f4450198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f4454cea640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450106f10 0x7f445019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 -- 172.21.15.46:0/1885600484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450106f10 msgr2=0x7f445019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4450106f10 0x7f445019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 -- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450102790 msgr2=0x7f4450198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4450102790 0x7f4450198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445019d3b0 con 0x7f4450101de0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444ffff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f4450198400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4440005940 tx=0x7f4440004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.875+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444003f030 con 0x7f4450101de0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.876+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4440003aa0 con 0x7f4450101de0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.876+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445019d640 con 0x7f4450101de0 2024-09-12T02:49:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.876+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440035070 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.876+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a4f80 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.877+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4450062900 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.878+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4440042030 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.878+0000 7f444dffb640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f443c0a2c90 0x7f443c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.878+0000 7f4447fff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f443c0a2c90 0x7f443c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.878+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f44400f0300 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.879+0000 7f444dffb640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] conn(0x7f443c0ade60 0x7f443c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.879+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f443c0b09d0 con 0x7f443c0ade60 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.879+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f44400f0750 con 0x7f4450101de0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.879+0000 7f4447fff640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f443c0a2c90 0x7f443c0a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44500fff00 tx=0x7f443800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.879+0000 7f4454cea640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] conn(0x7f443c0ade60 0x7f443c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.880+0000 7f4454cea640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] conn(0x7f443c0ade60 0x7f443c0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.880+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== osd.4 v2:172.21.15.145:6808/473009837 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f443c0b09d0 con 0x7f443c0ade60 2024-09-12T02:49:08.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.895+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 --> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4450062d00 con 0x7f443c0ade60 2024-09-12T02:49:08.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f444dffb640 1 -- 172.21.15.46:0/1885600484 <== osd.4 v2:172.21.15.145:6808/473009837 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4450062d00 con 0x7f443c0ade60 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] conn(0x7f443c0ade60 msgr2=0x7f443c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] conn(0x7f443c0ade60 0x7f443c0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f443c0a2c90 msgr2=0x7f443c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f443c0a2c90 0x7f443c0a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44500fff00 tx=0x7f443800b390 comp rx=0 tx=0).stop 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 msgr2=0x7f4450198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 --2- 172.21.15.46:0/1885600484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4450101de0 0x7f4450198400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4440005940 tx=0x7f4440004270 comp rx=0 tx=0).stop 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f4454cea640 1 -- 172.21.15.46:0/1885600484 reap_dead start 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 shutdown_connections 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 >> 172.21.15.46:0/1885600484 conn(0x7f44500fd8c0 msgr2=0x7f44500fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 shutdown_connections 2024-09-12T02:49:08.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:08.896+0000 7f445674d640 1 -- 172.21.15.46:0/1885600484 wait complete. 2024-09-12T02:49:09.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:09 smithi046 ceph-mon[30951]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:09.577 INFO:teuthology.orchestra.run.smithi046.stdout:244813135918 2024-09-12T02:49:09.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T02:49:09.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:09 smithi145 ceph-mon[34674]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:09.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:09 smithi196 ceph-mon[34587]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:09.781 INFO:teuthology.orchestra.run.smithi046.stdout:274877906984 2024-09-12T02:49:09.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T02:49:10.028 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:10.574 INFO:teuthology.orchestra.run.smithi046.stdout:141733920836 2024-09-12T02:49:10.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T02:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:111669149769 2024-09-12T02:49:11.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T02:49:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:11.135 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:11.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:11 smithi046 ceph-mon[30951]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:11 smithi145 ceph-mon[34674]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:11 smithi196 ceph-mon[34587]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:11.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 -- 172.21.15.46:0/3169152387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4100e20 msgr2=0x7fcad4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 --2- 172.21.15.46:0/3169152387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4100e20 0x7fcad4101200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009900 tx=0x7fcad002a480 comp rx=0 tx=0).stop 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 -- 172.21.15.46:0/3169152387 shutdown_connections 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 --2- 172.21.15.46:0/3169152387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 0x7fcad410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 --2- 172.21.15.46:0/3169152387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4102570 0x7fcad41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 --2- 172.21.15.46:0/3169152387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4100e20 0x7fcad4101200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 -- 172.21.15.46:0/3169152387 >> 172.21.15.46:0/3169152387 conn(0x7fcad40fc7d0 msgr2=0x7fcad40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 -- 172.21.15.46:0/3169152387 shutdown_connections 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.831+0000 7fcadae0d640 1 -- 172.21.15.46:0/3169152387 wait complete. 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 Processor -- start 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 -- start start 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4100e20 0x7fcad4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 0x7fcad419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4110e30 con 0x7fcad4102570 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4110cb0 con 0x7fcad4100e20 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcadae0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4110fb0 con 0x7fcad4102ef0 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49640/0 (socket says 172.21.15.46:49640) 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 -- 172.21.15.46:0/2951949236 learned_addr learned my addr 172.21.15.46:0/2951949236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcad8ba9640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4100e20 0x7fcad4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcad93aa640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 0x7fcad419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 -- 172.21.15.46:0/2951949236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 msgr2=0x7fcad419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 0x7fcad419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 -- 172.21.15.46:0/2951949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4100e20 msgr2=0x7fcad4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4100e20 0x7fcad4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.832+0000 7fcacbfff640 1 -- 172.21.15.46:0/2951949236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad419d440 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcacbfff640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d770 tx=0x7fcac400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4017070 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac40054d0 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad419d730 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4004cf0 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.833+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a5010 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.834+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad4101200 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.835+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcac4004e90 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.836+0000 7fcac9ffb640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 0x7fcabc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.836+0000 7fcad8ba9640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 0x7fcabc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.836+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcac40cefe0 con 0x7fcad4102570 2024-09-12T02:49:11.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.836+0000 7fcad8ba9640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 0x7fcabc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009230 tx=0x7fcad00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:11.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:11.844+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcac409ba40 con 0x7fcad4102570 2024-09-12T02:49:12.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.031+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcad410a0a0 con 0x7fcad4102570 2024-09-12T02:49:12.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.032+0000 7fcac9ffb640 1 -- 172.21.15.46:0/2951949236 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcac409b340 con 0x7fcad4102570 2024-09-12T02:49:12.034 INFO:teuthology.orchestra.run.smithi046.stdout:193273528377 2024-09-12T02:49:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 msgr2=0x7fcabc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 0x7fcabc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009230 tx=0x7fcad00058d0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 msgr2=0x7fcad41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d770 tx=0x7fcac400dc30 comp rx=0 tx=0).stop 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 shutdown_connections 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fcabc0a2c10 0x7fcabc0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad4102ef0 0x7fcad419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4102570 0x7fcad41989d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 --2- 172.21.15.46:0/2951949236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcad4100e20 0x7fcad4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 >> 172.21.15.46:0/2951949236 conn(0x7fcad40fc7d0 msgr2=0x7fcad406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 shutdown_connections 2024-09-12T02:49:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.034+0000 7fcadae0d640 1 -- 172.21.15.46:0/2951949236 wait complete. 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 -- 172.21.15.46:0/4245258234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 msgr2=0x7fbee410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/4245258234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee410ea90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009ca0 tx=0x7fbedc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 -- 172.21.15.46:0/4245258234 shutdown_connections 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/4245258234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee410ea90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/4245258234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 0x7fbee41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/4245258234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 0x7fbee41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 -- 172.21.15.46:0/4245258234 >> 172.21.15.46:0/4245258234 conn(0x7fbee40fd8c0 msgr2=0x7fbee40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 -- 172.21.15.46:0/4245258234 shutdown_connections 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.569+0000 7fbeec2c9640 1 -- 172.21.15.46:0/4245258234 wait complete. 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 Processor -- start 2024-09-12T02:49:12.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 -- start start 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 0x7fbee4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 0x7fbee4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4111e20 con 0x7fbee4106f10 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4111ca0 con 0x7fbee4102790 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeec2c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4111fa0 con 0x7fbee4101de0 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeea866640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeea866640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49656/0 (socket says 172.21.15.46:49656) 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeea866640 1 -- 172.21.15.46:0/3608344789 learned_addr learned my addr 172.21.15.46:0/3608344789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.570+0000 7fbeea065640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 0x7fbee4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbee9864640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 0x7fbee4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 -- 172.21.15.46:0/3608344789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 msgr2=0x7fbee4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 0x7fbee4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 -- 172.21.15.46:0/3608344789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 msgr2=0x7fbee4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 0x7fbee4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 -- 172.21.15.46:0/3608344789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee419d2d0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeea866640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009dc0 tx=0x7fbedc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc003cf0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee419d560 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbedc0044f0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc030bf0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.571+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41a4ea0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.572+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee41021c0 con 0x7fbee4106f10 2024-09-12T02:49:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.573+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbedc004040 con 0x7fbee4106f10 2024-09-12T02:49:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.574+0000 7fbedb7fe640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 0x7fbed00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.574+0000 7fbeea065640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 0x7fbed00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.574+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbedc0f0b40 con 0x7fbee4106f10 2024-09-12T02:49:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.575+0000 7fbeea065640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 0x7fbed00a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbee000bbe0 tx=0x7fbee000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.577+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbedc0bd6f0 con 0x7fbee4106f10 2024-09-12T02:49:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:12 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2951949236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:49:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:12 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2951949236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:49:12.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.761+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbee4199680 con 0x7fbee4106f10 2024-09-12T02:49:12.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:12 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2951949236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:49:12.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.761+0000 7fbedb7fe640 1 -- 172.21.15.46:0/3608344789 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbee4199680 con 0x7fbee4106f10 2024-09-12T02:49:12.763 INFO:teuthology.orchestra.run.smithi046.stdout:244813135920 2024-09-12T02:49:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 msgr2=0x7fbed00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 0x7fbed00a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbee000bbe0 tx=0x7fbee000b390 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 msgr2=0x7fbee419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009dc0 tx=0x7fbedc032ff0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 shutdown_connections 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee4106f10 0x7fbee419cbf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbee4102790 0x7fbee4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbed00a2c90 0x7fbed00a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 --2- 172.21.15.46:0/3608344789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee4101de0 0x7fbee4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 >> 172.21.15.46:0/3608344789 conn(0x7fbee40fd8c0 msgr2=0x7fbee40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 shutdown_connections 2024-09-12T02:49:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:12.764+0000 7fbeec2c9640 1 -- 172.21.15.46:0/3608344789 wait complete. 2024-09-12T02:49:13.397 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528374 got 193273528377 for osd.6 2024-09-12T02:49:13.397 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:13.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:13 smithi046 ceph-mon[30951]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:13.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:13 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3608344789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:49:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 -- 172.21.15.46:0/283195924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 msgr2=0x7f138410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 --2- 172.21.15.46:0/283195924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138410df40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1374009900 tx=0x7f137402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 -- 172.21.15.46:0/283195924 shutdown_connections 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 --2- 172.21.15.46:0/283195924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f1384110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 --2- 172.21.15.46:0/283195924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138410df40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 --2- 172.21.15.46:0/283195924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f1384100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 -- 172.21.15.46:0/283195924 >> 172.21.15.46:0/283195924 conn(0x7f13840fc7a0 msgr2=0x7f13840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.530+0000 7f138b704640 1 -- 172.21.15.46:0/283195924 shutdown_connections 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.531+0000 7f138b704640 1 -- 172.21.15.46:0/283195924 wait complete. 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.531+0000 7f138b704640 1 Processor -- start 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.531+0000 7f138b704640 1 -- start start 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.531+0000 7f138b704640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f138419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.531+0000 7f138b704640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f138b704640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f138419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f138b704640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114040 con 0x7f1384101280 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f138b704640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113ec0 con 0x7f138410e480 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f138b704640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13841141c0 con 0x7f13841008d0 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49676/0 (socket says 172.21.15.46:49676) 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 -- 172.21.15.46:0/4052579741 learned_addr learned my addr 172.21.15.46:0/4052579741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1389ca1640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f138419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f13894a0640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f138419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 -- 172.21.15.46:0/4052579741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 msgr2=0x7f138419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f138419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 -- 172.21.15.46:0/4052579741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 msgr2=0x7f138419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:13.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f138419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 -- 172.21.15.46:0/4052579741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138419f5b0 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f13894a0640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f138419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1389ca1640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f138419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.532+0000 7f1388c9f640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1374005940 tx=0x7f13740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.533+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137403f030 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.533+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1374003cb0 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.533+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374035070 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.533+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138419f840 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.533+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a7180 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.534+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1384100cb0 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.535+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1374042030 con 0x7f1384101280 2024-09-12T02:49:13.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.535+0000 7f137a7fc640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 0x7f13700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:13.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.536+0000 7f13894a0640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 0x7f13700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:13.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.536+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f13740efd10 con 0x7f1384101280 2024-09-12T02:49:13.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.536+0000 7f13894a0640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 0x7f13700a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f13800058c0 tx=0x7f1380005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:13.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.539+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13740bc7b0 con 0x7f1384101280 2024-09-12T02:49:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:13 smithi145 ceph-mon[34674]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:13.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:13 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3608344789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:49:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.718+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1384109610 con 0x7f1384101280 2024-09-12T02:49:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.718+0000 7f137a7fc640 1 -- 172.21.15.46:0/4052579741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13740bc0b0 con 0x7f1384101280 2024-09-12T02:49:13.720 INFO:teuthology.orchestra.run.smithi046.stdout:77309411407 2024-09-12T02:49:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:13 smithi196 ceph-mon[34587]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:13 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3608344789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:49:13.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.720+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 msgr2=0x7f13700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:13.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.720+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 0x7f13700a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f13800058c0 tx=0x7f1380005850 comp rx=0 tx=0).stop 2024-09-12T02:49:13.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.720+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 msgr2=0x7f138419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:13.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.720+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1374005940 tx=0x7f13740042f0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 shutdown_connections 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f138410e480 0x7f138419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384101280 0x7f138419ab40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f13700a2c10 0x7f13700a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 --2- 172.21.15.46:0/4052579741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13841008d0 0x7f138419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 >> 172.21.15.46:0/4052579741 conn(0x7f13840fc7a0 msgr2=0x7f13840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 shutdown_connections 2024-09-12T02:49:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:13.721+0000 7f138b704640 1 -- 172.21.15.46:0/4052579741 wait complete. 2024-09-12T02:49:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 -- 172.21.15.46:0/2131660878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102ef0 msgr2=0x7fef8c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 --2- 172.21.15.46:0/2131660878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102ef0 0x7fef8c10da40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402a950 comp rx=0 tx=0).stop 2024-09-12T02:49:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 -- 172.21.15.46:0/2131660878 shutdown_connections 2024-09-12T02:49:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 --2- 172.21.15.46:0/2131660878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102ef0 0x7fef8c10da40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 --2- 172.21.15.46:0/2131660878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c102570 0x7fef8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 --2- 172.21.15.46:0/2131660878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c100e20 0x7fef8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.354+0000 7fef918e2640 1 -- 172.21.15.46:0/2131660878 >> 172.21.15.46:0/2131660878 conn(0x7fef8c0fc7d0 msgr2=0x7fef8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.355+0000 7fef918e2640 1 -- 172.21.15.46:0/2131660878 shutdown_connections 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.355+0000 7fef918e2640 1 -- 172.21.15.46:0/2131660878 wait complete. 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 Processor -- start 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 -- start start 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c100e20 0x7fef8c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 0x7fef8c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110e30 con 0x7fef8c102570 2024-09-12T02:49:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110cb0 con 0x7fef8c102ef0 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef918e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110fb0 con 0x7fef8c100e20 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47976/0 (socket says 172.21.15.46:47976) 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef8a7fc640 1 -- 172.21.15.46:0/2047116515 learned_addr learned my addr 172.21.15.46:0/2047116515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef8b7fe640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 0x7fef8c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.356+0000 7fef8affd640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c100e20 0x7fef8c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 -- 172.21.15.46:0/2047116515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c100e20 msgr2=0x7fef8c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c100e20 0x7fef8c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 -- 172.21.15.46:0/2047116515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 msgr2=0x7fef8c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 0x7fef8c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 -- 172.21.15.46:0/2047116515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19f580 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8b7fe640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 0x7fef8c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.357+0000 7fef8a7fc640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e9f0 tx=0x7fef8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.358+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8000cd80 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.358+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c19f870 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.358+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef800108e0 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.358+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80018920 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.358+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1a7150 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.359+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c101200 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.360+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fef80010a90 con 0x7fef8c102570 2024-09-12T02:49:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.361+0000 7fef908e0640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 0x7fef740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.361+0000 7fef8affd640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 0x7fef740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.361+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fef800cfcb0 con 0x7fef8c102570 2024-09-12T02:49:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.361+0000 7fef8affd640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 0x7fef740a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009bd0 tx=0x7fef7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.368+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef8009c100 con 0x7fef8c102570 2024-09-12T02:49:14.423 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:14 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4052579741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:49:14.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 -- 172.21.15.46:0/4006110713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10106a80 msgr2=0x7f3f10106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/4006110713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10106a80 0x7f3f10106e60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009930 tx=0x7f3f0c02a420 comp rx=0 tx=0).stop 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 -- 172.21.15.46:0/4006110713 shutdown_connections 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/4006110713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101420 0x7f3f1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/4006110713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10100a80 0x7f3f10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/4006110713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10106a80 0x7f3f10106e60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 -- 172.21.15.46:0/4006110713 >> 172.21.15.46:0/4006110713 conn(0x7f3f100fc7a0 msgr2=0x7f3f100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 -- 172.21.15.46:0/4006110713 shutdown_connections 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.226+0000 7f3f17d0b640 1 -- 172.21.15.46:0/4006110713 wait complete. 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 Processor -- start 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 -- start start 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10100a80 0x7f3f10198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10106a80 0x7f3f1019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110d30 con 0x7f3f10101420 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110bb0 con 0x7f3f10106a80 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f17d0b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110eb0 con 0x7f3f10100a80 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47966/0 (socket says 172.21.15.46:47966) 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 -- 172.21.15.46:0/2148550324 learned_addr learned my addr 172.21.15.46:0/2148550324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f15aa7640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10100a80 0x7f3f10198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 -- 172.21.15.46:0/2148550324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10100a80 msgr2=0x7f3f10198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f162a8640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10106a80 0x7f3f1019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10100a80 0x7f3f10198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 -- 172.21.15.46:0/2148550324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10106a80 msgr2=0x7f3f1019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10106a80 0x7f3f1019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.227+0000 7f3f152a6640 1 -- 172.21.15.46:0/2148550324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1019d340 con 0x7f3f10101420 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stdout:163208757311 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f152a6640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d950 tx=0x7f3f0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00017070 con 0x7f3f10101420 2024-09-12T02:49:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f00009c30 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1019d630 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000f620 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.228+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a4f10 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.229+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10100ee0 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.230+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f000109a0 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.231+0000 7f3f06ffd640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 0x7f3efc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.231+0000 7f3f15aa7640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 0x7f3efc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.231+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3f000cf1f0 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.231+0000 7f3f15aa7640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 0x7f3efc0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000c00 tx=0x7f3f0c032ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.234+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f00009de0 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.421+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3f101a51d0 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.422+0000 7f3f06ffd640 1 -- 172.21.15.46:0/2148550324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3f0009bc90 con 0x7f3f10101420 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 msgr2=0x7f3efc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 0x7f3efc0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000c00 tx=0x7f3f0c032ba0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 msgr2=0x7f3f101988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d950 tx=0x7f3f0000de10 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 shutdown_connections 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f10106a80 0x7f3f1019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f10101420 0x7f3f101988d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3efc0a2c10 0x7f3efc0a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 --2- 172.21.15.46:0/2148550324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10100a80 0x7f3f10198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 >> 172.21.15.46:0/2148550324 conn(0x7f3f100fc7a0 msgr2=0x7f3f10104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 shutdown_connections 2024-09-12T02:49:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.424+0000 7f3f17d0b640 1 -- 172.21.15.46:0/2148550324 wait complete. 2024-09-12T02:49:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.548+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fef8c1a73e0 con 0x7fef8c102570 2024-09-12T02:49:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.549+0000 7fef908e0640 1 -- 172.21.15.46:0/2047116515 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef800a0400 con 0x7fef8c102570 2024-09-12T02:49:14.550 INFO:teuthology.orchestra.run.smithi046.stdout:34359738457 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 msgr2=0x7fef740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 0x7fef740a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009bd0 tx=0x7fef7c005f90 comp rx=0 tx=0).stop 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 msgr2=0x7fef8c19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e9f0 tx=0x7fef8000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 shutdown_connections 2024-09-12T02:49:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef8c102ef0 0x7fef8c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8c102570 0x7fef8c19ab10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fef740a2c10 0x7fef740a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 --2- 172.21.15.46:0/2047116515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef8c100e20 0x7fef8c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 >> 172.21.15.46:0/2047116515 conn(0x7fef8c0fc7d0 msgr2=0x7fef8c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 shutdown_connections 2024-09-12T02:49:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.551+0000 7fef918e2640 1 -- 172.21.15.46:0/2047116515 wait complete. 2024-09-12T02:49:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:14 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4052579741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:49:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:14 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4052579741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 -- 172.21.15.46:0/3005097955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0106f10 msgr2=0x7f63f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 --2- 172.21.15.46:0/3005097955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0106f10 0x7f63f010ea90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009ca0 tx=0x7f63e802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 -- 172.21.15.46:0/3005097955 shutdown_connections 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 --2- 172.21.15.46:0/3005097955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0106f10 0x7f63f010ea90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 --2- 172.21.15.46:0/3005097955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0102790 0x7f63f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 --2- 172.21.15.46:0/3005097955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0101de0 0x7f63f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 -- 172.21.15.46:0/3005097955 >> 172.21.15.46:0/3005097955 conn(0x7f63f00fd8c0 msgr2=0x7f63f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 -- 172.21.15.46:0/3005097955 shutdown_connections 2024-09-12T02:49:14.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.914+0000 7f63f68f4640 1 -- 172.21.15.46:0/3005097955 wait complete. 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 Processor -- start 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 -- start start 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 0x7f63f0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0106f10 0x7f63f019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0111e20 con 0x7f63f0102790 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0111ca0 con 0x7f63f0106f10 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63f68f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0111fa0 con 0x7f63f0101de0 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47996/0 (socket says 172.21.15.46:47996) 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.915+0000 7f63ef7fe640 1 -- 172.21.15.46:0/962387496 learned_addr learned my addr 172.21.15.46:0/962387496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63effff640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 0x7f63f0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63f4e91640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0106f10 0x7f63f019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 -- 172.21.15.46:0/962387496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 msgr2=0x7f63f0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 0x7f63f0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 -- 172.21.15.46:0/962387496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0106f10 msgr2=0x7f63f019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0106f10 0x7f63f019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 -- 172.21.15.46:0/962387496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f019d330 con 0x7f63f0102790 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63effff640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 0x7f63f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ef7fe640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63e400d770 tx=0x7f63e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4017070 con 0x7f63f0102790 2024-09-12T02:49:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63e40054d0 con 0x7f63f0102790 2024-09-12T02:49:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f019d620 con 0x7f63f0102790 2024-09-12T02:49:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4004cf0 con 0x7f63f0102790 2024-09-12T02:49:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.916+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a4f00 con 0x7f63f0102790 2024-09-12T02:49:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.917+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f01021c0 con 0x7f63f0102790 2024-09-12T02:49:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.919+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f63e4004e90 con 0x7f63f0102790 2024-09-12T02:49:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.919+0000 7f63ed7fa640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 0x7f63d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.920+0000 7f63effff640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 0x7f63d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:14.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.920+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f63e40cf200 con 0x7f63f0102790 2024-09-12T02:49:14.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.920+0000 7f63effff640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 0x7f63d80a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63e000bfd0 tx=0x7f63e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:14.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:14.923+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63e409b6d0 con 0x7f63f0102790 2024-09-12T02:49:15.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.100+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f63f01996e0 con 0x7f63f0102790 2024-09-12T02:49:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.102+0000 7f63ed7fa640 1 -- 172.21.15.46:0/962387496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63e409f9d0 con 0x7f63f0102790 2024-09-12T02:49:15.105 INFO:teuthology.orchestra.run.smithi046.stdout:55834574933 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.106+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 msgr2=0x7f63d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.106+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 0x7f63d80a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63e000bfd0 tx=0x7f63e000b390 comp rx=0 tx=0).stop 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.106+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 msgr2=0x7f63f01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.106+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63e400d770 tx=0x7f63e400dc30 comp rx=0 tx=0).stop 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 shutdown_connections 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0106f10 0x7f63f019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:15.109 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f0102790 0x7f63f01988c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f63d80a2c90 0x7f63d80a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 --2- 172.21.15.46:0/962387496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f0101de0 0x7f63f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.107+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 >> 172.21.15.46:0/962387496 conn(0x7f63f00fd8c0 msgr2=0x7f63f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.109+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 shutdown_connections 2024-09-12T02:49:15.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:15.109+0000 7f63f68f4640 1 -- 172.21.15.46:0/962387496 wait complete. 2024-09-12T02:49:15.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135918 got 244813135920 for osd.8 2024-09-12T02:49:15.584 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:15 smithi145 ceph-mon[34674]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:15 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2148550324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:49:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:15 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2047116515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:49:15.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:15 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/962387496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:49:15.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:15 smithi196 ceph-mon[34587]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:15 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2148550324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:49:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:15 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2047116515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:49:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:15 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/962387496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:49:15.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:15 smithi046 ceph-mon[30951]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:15.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:15 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2148550324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:49:15.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:15 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2047116515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:49:15.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:15 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/962387496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:49:16.408 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411407 for osd.2 2024-09-12T02:49:16.408 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:17.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757308 got 163208757311 for osd.5 2024-09-12T02:49:17.534 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:17 smithi196 ceph-mon[34587]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-12T02:49:17.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:17 smithi046 ceph-mon[30951]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-12T02:49:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:17 smithi145 ceph-mon[34674]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-12T02:49:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:18.166 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738457 for osd.0 2024-09-12T02:49:18.167 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:18.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574933 for osd.1 2024-09-12T02:49:18.556 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.737+0000 7f8ea370b640 1 -- 172.21.15.46:0/2415341991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 msgr2=0x7f8e9c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.737+0000 7f8ea370b640 1 --2- 172.21.15.46:0/2415341991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c100cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009900 tx=0x7f8e9802a410 comp rx=0 tx=0).stop 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 -- 172.21.15.46:0/2415341991 shutdown_connections 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 --2- 172.21.15.46:0/2415341991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 0x7f8e9c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 --2- 172.21.15.46:0/2415341991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 0x7f8e9c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 --2- 172.21.15.46:0/2415341991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c100cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 -- 172.21.15.46:0/2415341991 >> 172.21.15.46:0/2415341991 conn(0x7f8e9c0fc7a0 msgr2=0x7f8e9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 -- 172.21.15.46:0/2415341991 shutdown_connections 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 -- 172.21.15.46:0/2415341991 wait complete. 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 Processor -- start 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.738+0000 7f8ea370b640 1 -- start start 2024-09-12T02:49:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 0x7f8e9c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 0x7f8e9c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1140b0 con 0x7f8e9c1008d0 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c113f30 con 0x7f8e9c10e480 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea370b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c114230 con 0x7f8e9c101280 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48006/0 (socket says 172.21.15.46:48006) 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 -- 172.21.15.46:0/1690318986 learned_addr learned my addr 172.21.15.46:0/1690318986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea1ca8640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 0x7f8e9c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea0ca6640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 0x7f8e9c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 -- 172.21.15.46:0/1690318986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 msgr2=0x7f8e9c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 0x7f8e9c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 -- 172.21.15.46:0/1690318986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 msgr2=0x7f8e9c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 0x7f8e9c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 -- 172.21.15.46:0/1690318986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c19f600 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.739+0000 7f8ea14a7640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.740+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9803f030 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.740+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c19f890 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.740+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e980035f0 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.740+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98035070 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.740+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a71d0 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.741+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c100cb0 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.742+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e98003b10 con 0x7f8e9c1008d0 2024-09-12T02:49:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.743+0000 7f8e927fc640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 0x7f8e880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.743+0000 7f8ea0ca6640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 0x7f8e880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.743+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8e980effd0 con 0x7f8e9c1008d0 2024-09-12T02:49:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.743+0000 7f8ea0ca6640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 0x7f8e880a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c10e980 tx=0x7f8e8c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.747+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e980bca70 con 0x7f8e9c1008d0 2024-09-12T02:49:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.925+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8e9c109610 con 0x7f8e9c1008d0 2024-09-12T02:49:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.925+0000 7f8e927fc640 1 -- 172.21.15.46:0/1690318986 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e980bc370 con 0x7f8e9c1008d0 2024-09-12T02:49:18.926 INFO:teuthology.orchestra.run.smithi046.stdout:214748364854 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 msgr2=0x7f8e880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 0x7f8e880a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c10e980 tx=0x7f8e8c009110 comp rx=0 tx=0).stop 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 msgr2=0x7f8e9c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e98004270 comp rx=0 tx=0).stop 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 shutdown_connections 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e9c10e480 0x7f8e9c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8e880a2c10 0x7f8e880a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101280 0x7f8e9c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 --2- 172.21.15.46:0/1690318986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e9c1008d0 0x7f8e9c19a650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 >> 172.21.15.46:0/1690318986 conn(0x7f8e9c0fc7a0 msgr2=0x7f8e9c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:18.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 shutdown_connections 2024-09-12T02:49:18.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:18.927+0000 7f8ea370b640 1 -- 172.21.15.46:0/1690318986 wait complete. 2024-09-12T02:49:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.118+0000 7f62d1b9c640 1 -- 172.21.15.46:0/2655804199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc103780 msgr2=0x7f62cc107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.118+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/2655804199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc103780 0x7f62cc107bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009930 tx=0x7f62b402a420 comp rx=0 tx=0).stop 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.118+0000 7f62d1b9c640 1 -- 172.21.15.46:0/2655804199 shutdown_connections 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/2655804199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc108300 0x7f62cc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/2655804199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc103780 0x7f62cc107bc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/2655804199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 0x7f62cc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 -- 172.21.15.46:0/2655804199 >> 172.21.15.46:0/2655804199 conn(0x7f62cc0fc7d0 msgr2=0x7f62cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 -- 172.21.15.46:0/2655804199 shutdown_connections 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 -- 172.21.15.46:0/2655804199 wait complete. 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 Processor -- start 2024-09-12T02:49:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.119+0000 7f62d1b9c640 1 -- start start 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 0x7f62cc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc103780 0x7f62cc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc112000 con 0x7f62cc108300 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc111e80 con 0x7f62cc103780 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc112180 con 0x7f62cc102dd0 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48014/0 (socket says 172.21.15.46:48014) 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 -- 172.21.15.46:0/4163294793 learned_addr learned my addr 172.21.15.46:0/4163294793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62caffd640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc103780 0x7f62cc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cb7fe640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 0x7f62cc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 -- 172.21.15.46:0/4163294793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 msgr2=0x7f62cc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 0x7f62cc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 -- 172.21.15.46:0/4163294793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc103780 msgr2=0x7f62cc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc103780 0x7f62cc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 -- 172.21.15.46:0/4163294793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc19d600 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62cbfff640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009fd0 tx=0x7f62c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4011830 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c4016070 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc19d8f0 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4010de0 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.120+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1a51d0 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.121+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc103230 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.123+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f62c40119d0 con 0x7f62cc108300 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.124+0000 7f62c8ff9640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 0x7f62b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.124+0000 7f62cb7fe640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 0x7f62b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.124+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f62c40cf200 con 0x7f62cc108300 2024-09-12T02:49:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.124+0000 7f62cb7fe640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 0x7f62b80a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009f20 tx=0x7f62bc0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.127+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62c409bdb0 con 0x7f62cc108300 2024-09-12T02:49:19.144 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.311+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f62cc10a290 con 0x7f62cc108300 2024-09-12T02:49:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.311+0000 7f62c8ff9640 1 -- 172.21.15.46:0/4163294793 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f62c409b6b0 con 0x7f62cc108300 2024-09-12T02:49:19.312 INFO:teuthology.orchestra.run.smithi046.stdout:296352743462 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 msgr2=0x7f62b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 0x7f62b80a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009f20 tx=0x7f62bc0094f0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 msgr2=0x7f62cc19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009fd0 tx=0x7f62c400b8e0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 shutdown_connections 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62cc108300 0x7f62cc19cf20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62cc103780 0x7f62cc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f62b80a2c90 0x7f62b80a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 --2- 172.21.15.46:0/4163294793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62cc102dd0 0x7f62cc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.313+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 >> 172.21.15.46:0/4163294793 conn(0x7f62cc0fc7d0 msgr2=0x7f62cc105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.314+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 shutdown_connections 2024-09-12T02:49:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:19.314+0000 7f62d1b9c640 1 -- 172.21.15.46:0/4163294793 wait complete. 2024-09-12T02:49:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:19 smithi046 ceph-mon[30951]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T02:49:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:19 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1690318986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:49:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:19 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/4163294793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:49:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:19 smithi196 ceph-mon[34587]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T02:49:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:19 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1690318986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:49:19.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:19 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/4163294793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:49:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:19 smithi145 ceph-mon[34674]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T02:49:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1690318986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:49:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:19 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/4163294793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:49:20.121 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364851 got 214748364854 for osd.7 2024-09-12T02:49:20.121 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:20.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 -- 172.21.15.46:0/428520673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810c170 msgr2=0x7f3ee810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/428520673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810c170 0x7f3ee810e610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009ca0 tx=0x7f3ee002a950 comp rx=0 tx=0).stop 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 -- 172.21.15.46:0/428520673 shutdown_connections 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/428520673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810c170 0x7f3ee810e610 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/428520673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee8103660 0x7f3ee810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/428520673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8102d40 0x7f3ee8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 -- 172.21.15.46:0/428520673 >> 172.21.15.46:0/428520673 conn(0x7f3ee80fc7a0 msgr2=0x7f3ee80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 -- 172.21.15.46:0/428520673 shutdown_connections 2024-09-12T02:49:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.291+0000 7f3ef02d6640 1 -- 172.21.15.46:0/428520673 wait complete. 2024-09-12T02:49:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 Processor -- start 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 -- start start 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8103660 0x7f3ee81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 0x7f3ee819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8111e00 con 0x7f3ee8102d40 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8111c80 con 0x7f3ee8103660 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3ef02d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8111f80 con 0x7f3ee810c170 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3eee072640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3eee072640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48034/0 (socket says 172.21.15.46:48034) 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3eee072640 1 -- 172.21.15.46:0/1719149062 learned_addr learned my addr 172.21.15.46:0/1719149062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3eee873640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 0x7f3ee819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.292+0000 7f3eed871640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8103660 0x7f3ee81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 -- 172.21.15.46:0/1719149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 msgr2=0x7f3ee819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 0x7f3ee819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 -- 172.21.15.46:0/1719149062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8103660 msgr2=0x7f3ee81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8103660 0x7f3ee81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 -- 172.21.15.46:0/1719149062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee819d260 con 0x7f3ee8102d40 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee873640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 0x7f3ee819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:20.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3eee072640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c920 tx=0x7f3ee400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4007c70 con 0x7f3ee8102d40 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ee4007e10 con 0x7f3ee8102d40 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4018820 con 0x7f3ee8102d40 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee819d550 con 0x7f3ee8102d40 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.293+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81a4e30 con 0x7f3ee8102d40 2024-09-12T02:49:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.294+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee8103120 con 0x7f3ee8102d40 2024-09-12T02:49:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.295+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ee40189c0 con 0x7f3ee8102d40 2024-09-12T02:49:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.296+0000 7f3edf7fe640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.296+0000 7f3eed871640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.296+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3ee40cf6b0 con 0x7f3ee8102d40 2024-09-12T02:49:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.297+0000 7f3eed871640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 0x7f3ed40a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005d70 tx=0x7f3ed8005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.301+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee4014030 con 0x7f3ee8102d40 2024-09-12T02:49:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.479+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3ee8107240 con 0x7f3ee8102d40 2024-09-12T02:49:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.479+0000 7f3edf7fe640 1 -- 172.21.15.46:0/1719149062 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3ee400de20 con 0x7f3ee8102d40 2024-09-12T02:49:20.480 INFO:teuthology.orchestra.run.smithi046.stdout:141733920838 2024-09-12T02:49:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 msgr2=0x7f3ed40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 0x7f3ed40a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005d70 tx=0x7f3ed8005d00 comp rx=0 tx=0).stop 2024-09-12T02:49:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 msgr2=0x7f3ee81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c920 tx=0x7f3ee400cde0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 shutdown_connections 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ee810c170 0x7f3ee819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ee8103660 0x7f3ee81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 --2- 172.21.15.46:0/1719149062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee8102d40 0x7f3ee81982b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 >> 172.21.15.46:0/1719149062 conn(0x7f3ee80fc7a0 msgr2=0x7f3ee80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 shutdown_connections 2024-09-12T02:49:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.481+0000 7f3ef02d6640 1 -- 172.21.15.46:0/1719149062 wait complete. 2024-09-12T02:49:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- 172.21.15.46:0/2785736912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102570 msgr2=0x7fee6c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 --2- 172.21.15.46:0/2785736912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102570 0x7fee6c1029b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fee60009900 tx=0x7fee6002a480 comp rx=0 tx=0).stop 2024-09-12T02:49:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- 172.21.15.46:0/2785736912 shutdown_connections 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 --2- 172.21.15.46:0/2785736912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102ef0 0x7fee6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 --2- 172.21.15.46:0/2785736912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102570 0x7fee6c1029b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 --2- 172.21.15.46:0/2785736912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 0x7fee6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- 172.21.15.46:0/2785736912 >> 172.21.15.46:0/2785736912 conn(0x7fee6c0fc7d0 msgr2=0x7fee6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- 172.21.15.46:0/2785736912 shutdown_connections 2024-09-12T02:49:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- 172.21.15.46:0/2785736912 wait complete. 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 Processor -- start 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.915+0000 7fee735f9640 1 -- start start 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 0x7fee6c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102570 0x7fee6c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110e30 con 0x7fee6c102ef0 2024-09-12T02:49:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110cb0 con 0x7fee6c102570 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee735f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110fb0 con 0x7fee6c100e20 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48044/0 (socket says 172.21.15.46:48044) 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 -- 172.21.15.46:0/3453748760 learned_addr learned my addr 172.21.15.46:0/3453748760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71395640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 0x7fee6c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee70b94640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102570 0x7fee6c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 -- 172.21.15.46:0/3453748760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 msgr2=0x7fee6c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 0x7fee6c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 -- 172.21.15.46:0/3453748760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102570 msgr2=0x7fee6c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102570 0x7fee6c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 -- 172.21.15.46:0/3453748760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c19d400 con 0x7fee6c102ef0 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.916+0000 7fee71b96640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee6400eaa0 tx=0x7fee6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.917+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6400cdb0 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.917+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee64005020 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.917+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c19d6f0 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.917+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee640116d0 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.917+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1a4fd0 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.918+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c101200 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.919+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee640051d0 con 0x7fee6c102ef0 2024-09-12T02:49:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.920+0000 7fee5e7fc640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 0x7fee580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.920+0000 7fee71395640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 0x7fee580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.920+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fee640cfa50 con 0x7fee6c102ef0 2024-09-12T02:49:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.920+0000 7fee71395640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 0x7fee580a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee680058c0 tx=0x7fee6800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:20.924+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee64016030 con 0x7fee6c102ef0 2024-09-12T02:49:21.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.102+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fee6c1997b0 con 0x7fee6c102ef0 2024-09-12T02:49:21.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.102+0000 7fee5e7fc640 1 -- 172.21.15.46:0/3453748760 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fee6c1997b0 con 0x7fee6c102ef0 2024-09-12T02:49:21.103 INFO:teuthology.orchestra.run.smithi046.stdout:326417514530 2024-09-12T02:49:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.104+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 msgr2=0x7fee580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.104+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 0x7fee580a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee680058c0 tx=0x7fee6800a250 comp rx=0 tx=0).stop 2024-09-12T02:49:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.104+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 msgr2=0x7fee6c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.104+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee6400eaa0 tx=0x7fee6400ef60 comp rx=0 tx=0).stop 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 shutdown_connections 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee6c102ef0 0x7fee6c19cd20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee6c102570 0x7fee6c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fee580a2c10 0x7fee580a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 --2- 172.21.15.46:0/3453748760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee6c100e20 0x7fee6c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 >> 172.21.15.46:0/3453748760 conn(0x7fee6c0fc7d0 msgr2=0x7fee6c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 shutdown_connections 2024-09-12T02:49:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.105+0000 7fee735f9640 1 -- 172.21.15.46:0/3453748760 wait complete. 2024-09-12T02:49:21.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920838 for osd.4 2024-09-12T02:49:21.537 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:21.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:21 smithi145 ceph-mon[34674]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:21.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:21 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1719149062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:49:21.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:21 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3453748760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:49:21.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:21 smithi196 ceph-mon[34587]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:21.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:21 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1719149062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:49:21.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:21 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3453748760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:49:21.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:21 smithi046 ceph-mon[30951]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:21.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:21 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1719149062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:49:21.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:21 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3453748760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:49:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 -- 172.21.15.46:0/3558068621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 msgr2=0x7efc4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 --2- 172.21.15.46:0/3558068621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c1067e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc48009900 tx=0x7efc4802a410 comp rx=0 tx=0).stop 2024-09-12T02:49:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 -- 172.21.15.46:0/3558068621 shutdown_connections 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 --2- 172.21.15.46:0/3558068621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c106f10 0x7efc4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 --2- 172.21.15.46:0/3558068621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c1067e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 --2- 172.21.15.46:0/3558068621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c101de0 0x7efc4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 -- 172.21.15.46:0/3558068621 >> 172.21.15.46:0/3558068621 conn(0x7efc4c0fd8c0 msgr2=0x7efc4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 -- 172.21.15.46:0/3558068621 shutdown_connections 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 -- 172.21.15.46:0/3558068621 wait complete. 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.924+0000 7efc540d7640 1 Processor -- start 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 -- start start 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 0x7efc4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 0x7efc4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:21.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c111ec0 con 0x7efc4c102790 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c111d40 con 0x7efc4c106f10 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc540d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112040 con 0x7efc4c101de0 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48060/0 (socket says 172.21.15.46:48060) 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 -- 172.21.15.46:0/443861996 learned_addr learned my addr 172.21.15.46:0/443861996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 -- 172.21.15.46:0/443861996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 msgr2=0x7efc4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51e73640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 0x7efc4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc52674640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 0x7efc4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 0x7efc4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 -- 172.21.15.46:0/443861996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 msgr2=0x7efc4c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 0x7efc4c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 -- 172.21.15.46:0/443861996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c1a69e0 con 0x7efc4c102790 2024-09-12T02:49:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51e73640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 0x7efc4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc52674640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 0x7efc4c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc51672640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc48006010 tx=0x7efc48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.925+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4803f030 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.926+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc480035f0 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.926+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1a6c70 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.926+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48035070 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.926+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a71a0 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.927+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c102240 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.928+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc48003b10 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.928+0000 7efc3effd640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 0x7efc380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.929+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7efc480eff30 con 0x7efc4c102790 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.929+0000 7efc51e73640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 0x7efc380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.929+0000 7efc51e73640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 0x7efc380a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:21.933+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc480bc970 con 0x7efc4c102790 2024-09-12T02:49:22.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.112+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efc4c10a0f0 con 0x7efc4c102790 2024-09-12T02:49:22.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.113+0000 7efc3effd640 1 -- 172.21.15.46:0/443861996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc480bc270 con 0x7efc4c102790 2024-09-12T02:49:22.114 INFO:teuthology.orchestra.run.smithi046.stdout:111669149771 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 msgr2=0x7efc380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 0x7efc380a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40008040 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 msgr2=0x7efc4c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc48006010 tx=0x7efc48004270 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 shutdown_connections 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc4c106f10 0x7efc4c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c102790 0x7efc4c074a30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7efc380a2c90 0x7efc380a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 --2- 172.21.15.46:0/443861996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc4c101de0 0x7efc4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 >> 172.21.15.46:0/443861996 conn(0x7efc4c0fd8c0 msgr2=0x7efc4c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 shutdown_connections 2024-09-12T02:49:22.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:22.115+0000 7efc540d7640 1 -- 172.21.15.46:0/443861996 wait complete. 2024-09-12T02:49:22.542 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514526 got 326417514530 for osd.11 2024-09-12T02:49:22.543 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:22 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/443861996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:49:22.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:22 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/443861996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:49:22.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:22 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/443861996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:49:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:23.545 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743459 got 296352743462 for osd.10 2024-09-12T02:49:23.545 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:23.664 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:23 smithi046 ceph-mon[30951]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:23.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:23 smithi196 ceph-mon[34587]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:23.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:23 smithi145 ceph-mon[34674]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:49:24.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149769 got 111669149771 for osd.3 2024-09-12T02:49:24.269 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 -- 172.21.15.46:0/3653555066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 msgr2=0x7f4a901079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3653555066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a901079e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009900 tx=0x7f4a7802a410 comp rx=0 tx=0).stop 2024-09-12T02:49:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 -- 172.21.15.46:0/3653555066 shutdown_connections 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3653555066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a90108110 0x7f4a9010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3653555066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a901079e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3653555066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a901008e0 0x7f4a90100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 -- 172.21.15.46:0/3653555066 >> 172.21.15.46:0/3653555066 conn(0x7f4a90077210 msgr2=0x7f4a90077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 -- 172.21.15.46:0/3653555066 shutdown_connections 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.460+0000 7f4a9578d640 1 -- 172.21.15.46:0/3653555066 wait complete. 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 Processor -- start 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 -- start start 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a901008e0 0x7f4a901983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 0x7f4a9019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90111e60 con 0x7f4a90101290 2024-09-12T02:49:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90111ce0 con 0x7f4a90108110 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a9578d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90111fe0 con 0x7f4a901008e0 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a8e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a8e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49594/0 (socket says 172.21.15.46:49594) 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.461+0000 7f4a8e7fc640 1 -- 172.21.15.46:0/3200969006 learned_addr learned my addr 172.21.15.46:0/3200969006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8f7fe640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 0x7f4a9019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8effd640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a901008e0 0x7f4a901983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 -- 172.21.15.46:0/3200969006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a901008e0 msgr2=0x7f4a901983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a901008e0 0x7f4a901983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 -- 172.21.15.46:0/3200969006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 msgr2=0x7f4a9019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 0x7f4a9019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 -- 172.21.15.46:0/3200969006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9019d370 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8f7fe640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 0x7f4a9019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a8e7fc640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a78004270 tx=0x7f4a780042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7803f030 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9019d600 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a780044c0 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78035070 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.462+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a901a4f40 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.463+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a90100cc0 con 0x7f4a90101290 2024-09-12T02:49:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.464+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a78042030 con 0x7f4a90101290 2024-09-12T02:49:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.465+0000 7f4a87fff640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 0x7f4a7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.465+0000 7f4a8effd640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 0x7f4a7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.465+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4a780efbc0 con 0x7f4a90101290 2024-09-12T02:49:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.466+0000 7f4a8effd640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 0x7f4a7c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a80005c40 tx=0x7f4a8000a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.469+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a780bc600 con 0x7f4a90101290 2024-09-12T02:49:24.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.646+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4a9010a1c0 con 0x7f4a90101290 2024-09-12T02:49:24.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.646+0000 7f4a87fff640 1 -- 172.21.15.46:0/3200969006 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4a780bbf00 con 0x7f4a90101290 2024-09-12T02:49:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:274877906989 2024-09-12T02:49:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 msgr2=0x7f4a7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 0x7f4a7c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a80005c40 tx=0x7f4a8000a780 comp rx=0 tx=0).stop 2024-09-12T02:49:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 msgr2=0x7f4a90198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a78004270 tx=0x7f4a780042a0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 shutdown_connections 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90108110 0x7f4a9019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a90101290 0x7f4a90198900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f4a7c0a2c10 0x7f4a7c0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 --2- 172.21.15.46:0/3200969006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a901008e0 0x7f4a901983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.648+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 >> 172.21.15.46:0/3200969006 conn(0x7f4a90077210 msgr2=0x7f4a90105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.649+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 shutdown_connections 2024-09-12T02:49:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:24.649+0000 7f4a9578d640 1 -- 172.21.15.46:0/3200969006 wait complete. 2024-09-12T02:49:25.517 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906989 for osd.9 2024-09-12T02:49:25.517 DEBUG:teuthology.parallel:result is None 2024-09-12T02:49:25.518 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:49:25.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:49:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:25 smithi196 ceph-mon[34587]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:25 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3200969006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:49:25.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:25 smithi046 ceph-mon[30951]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:25 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3200969006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:49:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:25.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:25 smithi145 ceph-mon[34674]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:25.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:25 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3200969006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 -- 172.21.15.46:0/604872888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8102da0 msgr2=0x7f6fe8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/604872888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8102da0 0x7f6fe8103180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009900 tx=0x7f6fd802a480 comp rx=0 tx=0).stop 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 -- 172.21.15.46:0/604872888 shutdown_connections 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/604872888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe810c180 0x7f6fe810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/604872888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe81036c0 0x7f6fe810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/604872888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe8102da0 0x7f6fe8103180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 -- 172.21.15.46:0/604872888 >> 172.21.15.46:0/604872888 conn(0x7f6fe80fc7a0 msgr2=0x7f6fe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 -- 172.21.15.46:0/604872888 shutdown_connections 2024-09-12T02:49:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.309+0000 7f6fee2c4640 1 -- 172.21.15.46:0/604872888 wait complete. 2024-09-12T02:49:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 Processor -- start 2024-09-12T02:49:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 -- start start 2024-09-12T02:49:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 0x7f6fe81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 0x7f6fe819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8111e50 con 0x7f6fe81036c0 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8111cd0 con 0x7f6fe8102da0 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fee2c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8111fd0 con 0x7f6fe810c180 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49602/0 (socket says 172.21.15.46:49602) 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fdffff640 1 -- 172.21.15.46:0/4011986990 learned_addr learned my addr 172.21.15.46:0/4011986990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:26.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fec861640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 0x7f6fe819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.310+0000 7f6fe7fff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 0x7f6fe81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 -- 172.21.15.46:0/4011986990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 msgr2=0x7f6fe819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 0x7f6fe819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 -- 172.21.15.46:0/4011986990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 msgr2=0x7f6fe81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 0x7f6fe81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 -- 172.21.15.46:0/4011986990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe819d3a0 con 0x7f6fe81036c0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fec861640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 0x7f6fe819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fe7fff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 0x7f6fe81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fdffff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d950 tx=0x7f6fd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0017070 con 0x7f6fe81036c0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fd0009c30 con 0x7f6fe81036c0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe819d690 con 0x7f6fe81036c0 2024-09-12T02:49:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000f620 con 0x7f6fe81036c0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.311+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe81a4f70 con 0x7f6fe81036c0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.312+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe8103180 con 0x7f6fe81036c0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.313+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6fd00109a0 con 0x7f6fe81036c0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.314+0000 7f6fe5ffb640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 0x7f6fd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.314+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6fd00cf230 con 0x7f6fe81036c0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.314+0000 7f6fe7fff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 0x7f6fd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.314+0000 7f6fe7fff640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 0x7f6fd40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8031440 tx=0x7f6fd8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.318+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fd0009de0 con 0x7f6fe81036c0 2024-09-12T02:49:26.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.490+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fe8107260 con 0x7f6fd40a2c10 2024-09-12T02:49:26.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.499+0000 7f6fe5ffb640 1 -- 172.21.15.46:0/4011986990 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340585 (secure 0 0 0) 0x7f6fe8107260 con 0x7f6fd40a2c10 2024-09-12T02:49:26.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:49:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 msgr2=0x7f6fd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 0x7f6fd40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8031440 tx=0x7f6fd8005a20 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 msgr2=0x7f6fe8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d950 tx=0x7f6fd000de10 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 shutdown_connections 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6fd40a2c10 0x7f6fd40a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe810c180 0x7f6fe819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe81036c0 0x7f6fe8198930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 --2- 172.21.15.46:0/4011986990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fe8102da0 0x7f6fe81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 >> 172.21.15.46:0/4011986990 conn(0x7f6fe80fc7a0 msgr2=0x7f6fe80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 shutdown_connections 2024-09-12T02:49:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:26.502+0000 7f6fee2c4640 1 -- 172.21.15.46:0/4011986990 wait complete. 2024-09-12T02:49:27.265 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-12T02:49:26.287041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":540,"num_read_kb":394,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369644,"kb_used_data":12020,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572180,"statfs":{"total":1151940427776,"available":1151561912320,"internally_reserved":0,"allocated":12308480,"data_stored":7969037,"data_compressed":391518,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003325"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588213+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:37.588213+0000","last_peered":"2024-09-12T02:48:37.588213+0000","last_clean":"2024-09-12T02:48:37.588213+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:37.588213+0000","last_undegraded":"2024-09-12T02:48:37.588213+0000","last_fullsized":"2024-09-12T02:48:37.588213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339671+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:48:37.339671+0000","last_peered":"2024-09-12T02:48:37.339671+0000","last_clean":"2024-09-12T02:48:37.339671+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:48:37.339671+0000","last_undegraded":"2024-09-12T02:48:37.339671+0000","last_fullsized":"2024-09-12T02:48:37.339671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343048+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:48:37.343048+0000","last_peered":"2024-09-12T02:48:37.343048+0000","last_clean":"2024-09-12T02:48:37.343048+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:48:37.343048+0000","last_undegraded":"2024-09-12T02:48:37.343048+0000","last_fullsized":"2024-09-12T02:48:37.343048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339882+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:48:37.339882+0000","last_peered":"2024-09-12T02:48:37.339882+0000","last_clean":"2024-09-12T02:48:37.339882+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:48:37.339882+0000","last_undegraded":"2024-09-12T02:48:37.339882+0000","last_fullsized":"2024-09-12T02:48:37.339882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341105+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:48:37.341105+0000","last_peered":"2024-09-12T02:48:37.341105+0000","last_clean":"2024-09-12T02:48:37.341105+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:48:37.341105+0000","last_undegraded":"2024-09-12T02:48:37.341105+0000","last_fullsized":"2024-09-12T02:48:37.341105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340351+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:37.340351+0000","last_peered":"2024-09-12T02:48:37.340351+0000","last_clean":"2024-09-12T02:48:37.340351+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:37.340351+0000","last_undegraded":"2024-09-12T02:48:37.340351+0000","last_fullsized":"2024-09-12T02:48:37.340351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341138+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:37.341138+0000","last_peered":"2024-09-12T02:48:37.341138+0000","last_clean":"2024-09-12T02:48:37.341138+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:37.341138+0000","last_undegraded":"2024-09-12T02:48:37.341138+0000","last_fullsized":"2024-09-12T02:48:37.341138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339509+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:48:37.339509+0000","last_peered":"2024-09-12T02:48:37.339509+0000","last_clean":"2024-09-12T02:48:37.339509+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:48:37.339509+0000","last_undegraded":"2024-09-12T02:48:37.339509+0000","last_fullsized":"2024-09-12T02:48:37.339509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341184+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:37.341184+0000","last_peered":"2024-09-12T02:48:37.341184+0000","last_clean":"2024-09-12T02:48:37.341184+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:37.341184+0000","last_undegraded":"2024-09-12T02:48:37.341184+0000","last_fullsized":"2024-09-12T02:48:37.341184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340619+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:48:37.340619+0000","last_peered":"2024-09-12T02:48:37.340619+0000","last_clean":"2024-09-12T02:48:37.340619+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:48:37.340619+0000","last_undegraded":"2024-09-12T02:48:37.340619+0000","last_fullsized":"2024-09-12T02:48:37.340619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338203+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:37.338203+0000","last_peered":"2024-09-12T02:48:37.338203+0000","last_clean":"2024-09-12T02:48:37.338203+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:37.338203+0000","last_undegraded":"2024-09-12T02:48:37.338203+0000","last_fullsized":"2024-09-12T02:48:37.338203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340502+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:37.340502+0000","last_peered":"2024-09-12T02:48:37.340502+0000","last_clean":"2024-09-12T02:48:37.340502+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:37.340502+0000","last_undegraded":"2024-09-12T02:48:37.340502+0000","last_fullsized":"2024-09-12T02:48:37.340502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586858+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:48:37.586858+0000","last_peered":"2024-09-12T02:48:37.586858+0000","last_clean":"2024-09-12T02:48:37.586858+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:48:37.586858+0000","last_undegraded":"2024-09-12T02:48:37.586858+0000","last_fullsized":"2024-09-12T02:48:37.586858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339937+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:37.339937+0000","last_peered":"2024-09-12T02:48:37.339937+0000","last_clean":"2024-09-12T02:48:37.339937+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:37.339937+0000","last_undegraded":"2024-09-12T02:48:37.339937+0000","last_fullsized":"2024-09-12T02:48:37.339937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340671+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:37.340671+0000","last_peered":"2024-09-12T02:48:37.340671+0000","last_clean":"2024-09-12T02:48:37.340671+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:37.340671+0000","last_undegraded":"2024-09-12T02:48:37.340671+0000","last_fullsized":"2024-09-12T02:48:37.340671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339210+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:48:37.339210+0000","last_peered":"2024-09-12T02:48:37.339210+0000","last_clean":"2024-09-12T02:48:37.339210+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:48:37.339210+0000","last_undegraded":"2024-09-12T02:48:37.339210+0000","last_fullsized":"2024-09-12T02:48:37.339210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343983+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:48:37.343983+0000","last_peered":"2024-09-12T02:48:37.343983+0000","last_clean":"2024-09-12T02:48:37.343983+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:48:37.343983+0000","last_undegraded":"2024-09-12T02:48:37.343983+0000","last_fullsized":"2024-09-12T02:48:37.343983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341063+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:37.341063+0000","last_peered":"2024-09-12T02:48:37.341063+0000","last_clean":"2024-09-12T02:48:37.341063+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:37.341063+0000","last_undegraded":"2024-09-12T02:48:37.341063+0000","last_fullsized":"2024-09-12T02:48:37.341063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340770+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:48:37.340770+0000","last_peered":"2024-09-12T02:48:37.340770+0000","last_clean":"2024-09-12T02:48:37.340770+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:48:37.340770+0000","last_undegraded":"2024-09-12T02:48:37.340770+0000","last_fullsized":"2024-09-12T02:48:37.340770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339382+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:37.339382+0000","last_peered":"2024-09-12T02:48:37.339382+0000","last_clean":"2024-09-12T02:48:37.339382+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:37.339382+0000","last_undegraded":"2024-09-12T02:48:37.339382+0000","last_fullsized":"2024-09-12T02:48:37.339382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588035+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:37.588035+0000","last_peered":"2024-09-12T02:48:37.588035+0000","last_clean":"2024-09-12T02:48:37.588035+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:37.588035+0000","last_undegraded":"2024-09-12T02:48:37.588035+0000","last_fullsized":"2024-09-12T02:48:37.588035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339398+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:48:37.339398+0000","last_peered":"2024-09-12T02:48:37.339398+0000","last_clean":"2024-09-12T02:48:37.339398+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:48:37.339398+0000","last_undegraded":"2024-09-12T02:48:37.339398+0000","last_fullsized":"2024-09-12T02:48:37.339398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339366+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:37.339366+0000","last_peered":"2024-09-12T02:48:37.339366+0000","last_clean":"2024-09-12T02:48:37.339366+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:37.339366+0000","last_undegraded":"2024-09-12T02:48:37.339366+0000","last_fullsized":"2024-09-12T02:48:37.339366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.590486+0000","last_change":"2024-09-12T02:48:37.590486+0000","last_active":"2024-09-12T02:48:37.590486+0000","last_peered":"2024-09-12T02:48:37.590486+0000","last_clean":"2024-09-12T02:48:37.590486+0000","last_became_active":"2024-09-12T02:48:37.589930+0000","last_became_peered":"2024-09-12T02:48:37.589930+0000","last_unstale":"2024-09-12T02:48:37.590486+0000","last_undegraded":"2024-09-12T02:48:37.590486+0000","last_fullsized":"2024-09-12T02:48:37.590486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:42.323965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341047+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:48:37.341047+0000","last_peered":"2024-09-12T02:48:37.341047+0000","last_clean":"2024-09-12T02:48:37.341047+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:48:37.341047+0000","last_undegraded":"2024-09-12T02:48:37.341047+0000","last_fullsized":"2024-09-12T02:48:37.341047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340953+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:37.340953+0000","last_peered":"2024-09-12T02:48:37.340953+0000","last_clean":"2024-09-12T02:48:37.340953+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:37.340953+0000","last_undegraded":"2024-09-12T02:48:37.340953+0000","last_fullsized":"2024-09-12T02:48:37.340953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343184+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:48:37.343184+0000","last_peered":"2024-09-12T02:48:37.343184+0000","last_clean":"2024-09-12T02:48:37.343184+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:48:37.343184+0000","last_undegraded":"2024-09-12T02:48:37.343184+0000","last_fullsized":"2024-09-12T02:48:37.343184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339781+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:48:37.339781+0000","last_peered":"2024-09-12T02:48:37.339781+0000","last_clean":"2024-09-12T02:48:37.339781+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:48:37.339781+0000","last_undegraded":"2024-09-12T02:48:37.339781+0000","last_fullsized":"2024-09-12T02:48:37.339781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588143+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:37.588143+0000","last_peered":"2024-09-12T02:48:37.588143+0000","last_clean":"2024-09-12T02:48:37.588143+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:37.588143+0000","last_undegraded":"2024-09-12T02:48:37.588143+0000","last_fullsized":"2024-09-12T02:48:37.588143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339449+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:48:37.339449+0000","last_peered":"2024-09-12T02:48:37.339449+0000","last_clean":"2024-09-12T02:48:37.339449+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:48:37.339449+0000","last_undegraded":"2024-09-12T02:48:37.339449+0000","last_fullsized":"2024-09-12T02:48:37.339449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336705+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:48:37.336705+0000","last_peered":"2024-09-12T02:48:37.336705+0000","last_clean":"2024-09-12T02:48:37.336705+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:48:37.336705+0000","last_undegraded":"2024-09-12T02:48:37.336705+0000","last_fullsized":"2024-09-12T02:48:37.336705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341250+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:37.341250+0000","last_peered":"2024-09-12T02:48:37.341250+0000","last_clean":"2024-09-12T02:48:37.341250+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:37.341250+0000","last_undegraded":"2024-09-12T02:48:37.341250+0000","last_fullsized":"2024-09-12T02:48:37.341250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:48:37.341552+0000","last_peered":"2024-09-12T02:48:37.341552+0000","last_clean":"2024-09-12T02:48:37.341552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:48:37.341552+0000","last_undegraded":"2024-09-12T02:48:37.341552+0000","last_fullsized":"2024-09-12T02:48:37.341552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343136+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:48:37.343136+0000","last_peered":"2024-09-12T02:48:37.343136+0000","last_clean":"2024-09-12T02:48:37.343136+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:48:37.343136+0000","last_undegraded":"2024-09-12T02:48:37.343136+0000","last_fullsized":"2024-09-12T02:48:37.343136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587950+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:37.587950+0000","last_peered":"2024-09-12T02:48:37.587950+0000","last_clean":"2024-09-12T02:48:37.587950+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:37.587950+0000","last_undegraded":"2024-09-12T02:48:37.587950+0000","last_fullsized":"2024-09-12T02:48:37.587950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339541+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:48:37.339541+0000","last_peered":"2024-09-12T02:48:37.339541+0000","last_clean":"2024-09-12T02:48:37.339541+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:48:37.339541+0000","last_undegraded":"2024-09-12T02:48:37.339541+0000","last_fullsized":"2024-09-12T02:48:37.339541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588702+0000","last_change":"2024-09-12T02:48:37.588702+0000","last_active":"2024-09-12T02:48:37.588702+0000","last_peered":"2024-09-12T02:48:37.588702+0000","last_clean":"2024-09-12T02:48:37.588702+0000","last_became_active":"2024-09-12T02:48:37.588347+0000","last_became_peered":"2024-09-12T02:48:37.588347+0000","last_unstale":"2024-09-12T02:48:37.588702+0000","last_undegraded":"2024-09-12T02:48:37.588702+0000","last_fullsized":"2024-09-12T02:48:37.588702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:30.609070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586990+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:48:37.586990+0000","last_peered":"2024-09-12T02:48:37.586990+0000","last_clean":"2024-09-12T02:48:37.586990+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:48:37.586990+0000","last_undegraded":"2024-09-12T02:48:37.586990+0000","last_fullsized":"2024-09-12T02:48:37.586990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336464+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:48:37.336464+0000","last_peered":"2024-09-12T02:48:37.336464+0000","last_clean":"2024-09-12T02:48:37.336464+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:48:37.336464+0000","last_undegraded":"2024-09-12T02:48:37.336464+0000","last_fullsized":"2024-09-12T02:48:37.336464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337504+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:37.337504+0000","last_peered":"2024-09-12T02:48:37.337504+0000","last_clean":"2024-09-12T02:48:37.337504+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:37.337504+0000","last_undegraded":"2024-09-12T02:48:37.337504+0000","last_fullsized":"2024-09-12T02:48:37.337504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:23.194259+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:49:23.194259+0000","last_peered":"2024-09-12T02:49:23.194259+0000","last_clean":"2024-09-12T02:49:23.194259+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:49:23.194259+0000","last_undegraded":"2024-09-12T02:49:23.194259+0000","last_fullsized":"2024-09-12T02:49:23.194259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336272+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:48:37.336272+0000","last_peered":"2024-09-12T02:48:37.336272+0000","last_clean":"2024-09-12T02:48:37.336272+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:48:37.336272+0000","last_undegraded":"2024-09-12T02:48:37.336272+0000","last_fullsized":"2024-09-12T02:48:37.336272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588802+0000","last_change":"2024-09-12T02:48:37.588802+0000","last_active":"2024-09-12T02:48:37.588802+0000","last_peered":"2024-09-12T02:48:37.588802+0000","last_clean":"2024-09-12T02:48:37.588802+0000","last_became_active":"2024-09-12T02:48:37.588326+0000","last_became_peered":"2024-09-12T02:48:37.588326+0000","last_unstale":"2024-09-12T02:48:37.588802+0000","last_undegraded":"2024-09-12T02:48:37.588802+0000","last_fullsized":"2024-09-12T02:48:37.588802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586758+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:37.586758+0000","last_peered":"2024-09-12T02:48:37.586758+0000","last_clean":"2024-09-12T02:48:37.586758+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:37.586758+0000","last_undegraded":"2024-09-12T02:48:37.586758+0000","last_fullsized":"2024-09-12T02:48:37.586758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345128+0000","last_change":"2024-09-12T02:48:37.345128+0000","last_active":"2024-09-12T02:48:37.345128+0000","last_peered":"2024-09-12T02:48:37.345128+0000","last_clean":"2024-09-12T02:48:37.345128+0000","last_became_active":"2024-09-12T02:48:37.344670+0000","last_became_peered":"2024-09-12T02:48:37.344670+0000","last_unstale":"2024-09-12T02:48:37.345128+0000","last_undegraded":"2024-09-12T02:48:37.345128+0000","last_fullsized":"2024-09-12T02:48:37.345128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:32.206800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339329+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:48:37.339329+0000","last_peered":"2024-09-12T02:48:37.339329+0000","last_clean":"2024-09-12T02:48:37.339329+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:48:37.339329+0000","last_undegraded":"2024-09-12T02:48:37.339329+0000","last_fullsized":"2024-09-12T02:48:37.339329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341367+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:37.341367+0000","last_peered":"2024-09-12T02:48:37.341367+0000","last_clean":"2024-09-12T02:48:37.341367+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:37.341367+0000","last_undegraded":"2024-09-12T02:48:37.341367+0000","last_fullsized":"2024-09-12T02:48:37.341367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586962+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:48:37.586962+0000","last_peered":"2024-09-12T02:48:37.586962+0000","last_clean":"2024-09-12T02:48:37.586962+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:48:37.586962+0000","last_undegraded":"2024-09-12T02:48:37.586962+0000","last_fullsized":"2024-09-12T02:48:37.586962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586973+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:48:37.586973+0000","last_peered":"2024-09-12T02:48:37.586973+0000","last_clean":"2024-09-12T02:48:37.586973+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:48:37.586973+0000","last_undegraded":"2024-09-12T02:48:37.586973+0000","last_fullsized":"2024-09-12T02:48:37.586973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587180+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:48:37.587180+0000","last_peered":"2024-09-12T02:48:37.587180+0000","last_clean":"2024-09-12T02:48:37.587180+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:48:37.587180+0000","last_undegraded":"2024-09-12T02:48:37.587180+0000","last_fullsized":"2024-09-12T02:48:37.587180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343649+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:37.343649+0000","last_peered":"2024-09-12T02:48:37.343649+0000","last_clean":"2024-09-12T02:48:37.343649+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:37.343649+0000","last_undegraded":"2024-09-12T02:48:37.343649+0000","last_fullsized":"2024-09-12T02:48:37.343649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339501+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:48:37.339501+0000","last_peered":"2024-09-12T02:48:37.339501+0000","last_clean":"2024-09-12T02:48:37.339501+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:48:37.339501+0000","last_undegraded":"2024-09-12T02:48:37.339501+0000","last_fullsized":"2024-09-12T02:48:37.339501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587059+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:48:37.587059+0000","last_peered":"2024-09-12T02:48:37.587059+0000","last_clean":"2024-09-12T02:48:37.587059+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:48:37.587059+0000","last_undegraded":"2024-09-12T02:48:37.587059+0000","last_fullsized":"2024-09-12T02:48:37.587059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338318+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:37.338318+0000","last_peered":"2024-09-12T02:48:37.338318+0000","last_clean":"2024-09-12T02:48:37.338318+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:37.338318+0000","last_undegraded":"2024-09-12T02:48:37.338318+0000","last_fullsized":"2024-09-12T02:48:37.338318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.346098+0000","last_change":"2024-09-12T02:48:37.346098+0000","last_active":"2024-09-12T02:48:37.346098+0000","last_peered":"2024-09-12T02:48:37.346098+0000","last_clean":"2024-09-12T02:48:37.346098+0000","last_became_active":"2024-09-12T02:48:37.345696+0000","last_became_peered":"2024-09-12T02:48:37.345696+0000","last_unstale":"2024-09-12T02:48:37.346098+0000","last_undegraded":"2024-09-12T02:48:37.346098+0000","last_fullsized":"2024-09-12T02:48:37.346098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:48.040441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587347+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:48:37.587347+0000","last_peered":"2024-09-12T02:48:37.587347+0000","last_clean":"2024-09-12T02:48:37.587347+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:48:37.587347+0000","last_undegraded":"2024-09-12T02:48:37.587347+0000","last_fullsized":"2024-09-12T02:48:37.587347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336481+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:48:37.336481+0000","last_peered":"2024-09-12T02:48:37.336481+0000","last_clean":"2024-09-12T02:48:37.336481+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:48:37.336481+0000","last_undegraded":"2024-09-12T02:48:37.336481+0000","last_fullsized":"2024-09-12T02:48:37.336481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337296+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:37.337296+0000","last_peered":"2024-09-12T02:48:37.337296+0000","last_clean":"2024-09-12T02:48:37.337296+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:37.337296+0000","last_undegraded":"2024-09-12T02:48:37.337296+0000","last_fullsized":"2024-09-12T02:48:37.337296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341055+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:48:37.341055+0000","last_peered":"2024-09-12T02:48:37.341055+0000","last_clean":"2024-09-12T02:48:37.341055+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:48:37.341055+0000","last_undegraded":"2024-09-12T02:48:37.341055+0000","last_fullsized":"2024-09-12T02:48:37.341055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340312+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:37.340312+0000","last_peered":"2024-09-12T02:48:37.340312+0000","last_clean":"2024-09-12T02:48:37.340312+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:37.340312+0000","last_undegraded":"2024-09-12T02:48:37.340312+0000","last_fullsized":"2024-09-12T02:48:37.340312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337321+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:37.337321+0000","last_peered":"2024-09-12T02:48:37.337321+0000","last_clean":"2024-09-12T02:48:37.337321+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:37.337321+0000","last_undegraded":"2024-09-12T02:48:37.337321+0000","last_fullsized":"2024-09-12T02:48:37.337321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337404+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:37.337404+0000","last_peered":"2024-09-12T02:48:37.337404+0000","last_clean":"2024-09-12T02:48:37.337404+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:37.337404+0000","last_undegraded":"2024-09-12T02:48:37.337404+0000","last_fullsized":"2024-09-12T02:48:37.337404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339395+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:37.339395+0000","last_peered":"2024-09-12T02:48:37.339395+0000","last_clean":"2024-09-12T02:48:37.339395+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:37.339395+0000","last_undegraded":"2024-09-12T02:48:37.339395+0000","last_fullsized":"2024-09-12T02:48:37.339395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.344145+0000","last_change":"2024-09-12T02:48:37.344145+0000","last_active":"2024-09-12T02:48:37.344145+0000","last_peered":"2024-09-12T02:48:37.344145+0000","last_clean":"2024-09-12T02:48:37.344145+0000","last_became_active":"2024-09-12T02:48:37.343694+0000","last_became_peered":"2024-09-12T02:48:37.343694+0000","last_unstale":"2024-09-12T02:48:37.344145+0000","last_undegraded":"2024-09-12T02:48:37.344145+0000","last_fullsized":"2024-09-12T02:48:37.344145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:55.742228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586644+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:48:37.586644+0000","last_peered":"2024-09-12T02:48:37.586644+0000","last_clean":"2024-09-12T02:48:37.586644+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:48:37.586644+0000","last_undegraded":"2024-09-12T02:48:37.586644+0000","last_fullsized":"2024-09-12T02:48:37.586644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":101,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:20.190639+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:49:20.190639+0000","last_peered":"2024-09-12T02:49:20.190639+0000","last_clean":"2024-09-12T02:49:20.190639+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:49:20.190639+0000","last_undegraded":"2024-09-12T02:49:20.190639+0000","last_fullsized":"2024-09-12T02:49:20.190639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587962+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:37.587962+0000","last_peered":"2024-09-12T02:48:37.587962+0000","last_clean":"2024-09-12T02:48:37.587962+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:37.587962+0000","last_undegraded":"2024-09-12T02:48:37.587962+0000","last_fullsized":"2024-09-12T02:48:37.587962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587055+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:48:37.587055+0000","last_peered":"2024-09-12T02:48:37.587055+0000","last_clean":"2024-09-12T02:48:37.587055+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:48:37.587055+0000","last_undegraded":"2024-09-12T02:48:37.587055+0000","last_fullsized":"2024-09-12T02:48:37.587055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337741+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:37.337741+0000","last_peered":"2024-09-12T02:48:37.337741+0000","last_clean":"2024-09-12T02:48:37.337741+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:37.337741+0000","last_undegraded":"2024-09-12T02:48:37.337741+0000","last_fullsized":"2024-09-12T02:48:37.337741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341156+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:48:37.341156+0000","last_peered":"2024-09-12T02:48:37.341156+0000","last_clean":"2024-09-12T02:48:37.341156+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:48:37.341156+0000","last_undegraded":"2024-09-12T02:48:37.341156+0000","last_fullsized":"2024-09-12T02:48:37.341156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336133+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:48:37.336133+0000","last_peered":"2024-09-12T02:48:37.336133+0000","last_clean":"2024-09-12T02:48:37.336133+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:48:37.336133+0000","last_undegraded":"2024-09-12T02:48:37.336133+0000","last_fullsized":"2024-09-12T02:48:37.336133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.353056+0000","last_change":"2024-09-12T02:48:37.352674+0000","last_active":"2024-09-12T02:48:37.353056+0000","last_peered":"2024-09-12T02:48:37.353056+0000","last_clean":"2024-09-12T02:48:37.353056+0000","last_became_active":"2024-09-12T02:48:37.345293+0000","last_became_peered":"2024-09-12T02:48:37.345293+0000","last_unstale":"2024-09-12T02:48:37.353056+0000","last_undegraded":"2024-09-12T02:48:37.353056+0000","last_fullsized":"2024-09-12T02:48:37.353056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:00.609598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338121+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:37.338121+0000","last_peered":"2024-09-12T02:48:37.338121+0000","last_clean":"2024-09-12T02:48:37.338121+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:37.338121+0000","last_undegraded":"2024-09-12T02:48:37.338121+0000","last_fullsized":"2024-09-12T02:48:37.338121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340161+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:48:37.340161+0000","last_peered":"2024-09-12T02:48:37.340161+0000","last_clean":"2024-09-12T02:48:37.340161+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:48:37.340161+0000","last_undegraded":"2024-09-12T02:48:37.340161+0000","last_fullsized":"2024-09-12T02:48:37.340161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:21.191278+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:49:21.191278+0000","last_peered":"2024-09-12T02:49:21.191278+0000","last_clean":"2024-09-12T02:49:21.191278+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:49:21.191278+0000","last_undegraded":"2024-09-12T02:49:21.191278+0000","last_fullsized":"2024-09-12T02:49:21.191278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":133,"num_read_kb":129,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587651+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:37.587651+0000","last_peered":"2024-09-12T02:48:37.587651+0000","last_clean":"2024-09-12T02:48:37.587651+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:37.587651+0000","last_undegraded":"2024-09-12T02:48:37.587651+0000","last_fullsized":"2024-09-12T02:48:37.587651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339610+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:48:37.339610+0000","last_peered":"2024-09-12T02:48:37.339610+0000","last_clean":"2024-09-12T02:48:37.339610+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:48:37.339610+0000","last_undegraded":"2024-09-12T02:48:37.339610+0000","last_fullsized":"2024-09-12T02:48:37.339610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338294+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:37.338294+0000","last_peered":"2024-09-12T02:48:37.338294+0000","last_clean":"2024-09-12T02:48:37.338294+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:37.338294+0000","last_undegraded":"2024-09-12T02:48:37.338294+0000","last_fullsized":"2024-09-12T02:48:37.338294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341022+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:37.341022+0000","last_peered":"2024-09-12T02:48:37.341022+0000","last_clean":"2024-09-12T02:48:37.341022+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:37.341022+0000","last_undegraded":"2024-09-12T02:48:37.341022+0000","last_fullsized":"2024-09-12T02:48:37.341022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341169+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:48:37.341169+0000","last_peered":"2024-09-12T02:48:37.341169+0000","last_clean":"2024-09-12T02:48:37.341169+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:48:37.341169+0000","last_undegraded":"2024-09-12T02:48:37.341169+0000","last_fullsized":"2024-09-12T02:48:37.341169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587207+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:37.587207+0000","last_peered":"2024-09-12T02:48:37.587207+0000","last_clean":"2024-09-12T02:48:37.587207+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:37.587207+0000","last_undegraded":"2024-09-12T02:48:37.587207+0000","last_fullsized":"2024-09-12T02:48:37.587207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587311+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:37.587311+0000","last_peered":"2024-09-12T02:48:37.587311+0000","last_clean":"2024-09-12T02:48:37.587311+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:37.587311+0000","last_undegraded":"2024-09-12T02:48:37.587311+0000","last_fullsized":"2024-09-12T02:48:37.587311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340162+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:48:37.340162+0000","last_peered":"2024-09-12T02:48:37.340162+0000","last_clean":"2024-09-12T02:48:37.340162+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:48:37.340162+0000","last_undegraded":"2024-09-12T02:48:37.340162+0000","last_fullsized":"2024-09-12T02:48:37.340162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:20.189847+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:49:20.189847+0000","last_peered":"2024-09-12T02:49:20.189847+0000","last_clean":"2024-09-12T02:49:20.189847+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:49:20.189847+0000","last_undegraded":"2024-09-12T02:49:20.189847+0000","last_fullsized":"2024-09-12T02:49:20.189847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339163+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:48:37.339163+0000","last_peered":"2024-09-12T02:48:37.339163+0000","last_clean":"2024-09-12T02:48:37.339163+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:48:37.339163+0000","last_undegraded":"2024-09-12T02:48:37.339163+0000","last_fullsized":"2024-09-12T02:48:37.339163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340919+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:48:37.340919+0000","last_peered":"2024-09-12T02:48:37.340919+0000","last_clean":"2024-09-12T02:48:37.340919+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:48:37.340919+0000","last_undegraded":"2024-09-12T02:48:37.340919+0000","last_fullsized":"2024-09-12T02:48:37.340919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587478+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:37.587478+0000","last_peered":"2024-09-12T02:48:37.587478+0000","last_clean":"2024-09-12T02:48:37.587478+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:37.587478+0000","last_undegraded":"2024-09-12T02:48:37.587478+0000","last_fullsized":"2024-09-12T02:48:37.587478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337681+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:37.337681+0000","last_peered":"2024-09-12T02:48:37.337681+0000","last_clean":"2024-09-12T02:48:37.337681+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:37.337681+0000","last_undegraded":"2024-09-12T02:48:37.337681+0000","last_fullsized":"2024-09-12T02:48:37.337681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340797+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:48:37.340797+0000","last_peered":"2024-09-12T02:48:37.340797+0000","last_clean":"2024-09-12T02:48:37.340797+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:48:37.340797+0000","last_undegraded":"2024-09-12T02:48:37.340797+0000","last_fullsized":"2024-09-12T02:48:37.340797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339975+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:48:37.339975+0000","last_peered":"2024-09-12T02:48:37.339975+0000","last_clean":"2024-09-12T02:48:37.339975+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:48:37.339975+0000","last_undegraded":"2024-09-12T02:48:37.339975+0000","last_fullsized":"2024-09-12T02:48:37.339975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336024+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:37.336024+0000","last_peered":"2024-09-12T02:48:37.336024+0000","last_clean":"2024-09-12T02:48:37.336024+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:37.336024+0000","last_undegraded":"2024-09-12T02:48:37.336024+0000","last_fullsized":"2024-09-12T02:48:37.336024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343029+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:37.343029+0000","last_peered":"2024-09-12T02:48:37.343029+0000","last_clean":"2024-09-12T02:48:37.343029+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:37.343029+0000","last_undegraded":"2024-09-12T02:48:37.343029+0000","last_fullsized":"2024-09-12T02:48:37.343029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337742+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:37.337742+0000","last_peered":"2024-09-12T02:48:37.337742+0000","last_clean":"2024-09-12T02:48:37.337742+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:37.337742+0000","last_undegraded":"2024-09-12T02:48:37.337742+0000","last_fullsized":"2024-09-12T02:48:37.337742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343312+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:48:37.343312+0000","last_peered":"2024-09-12T02:48:37.343312+0000","last_clean":"2024-09-12T02:48:37.343312+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:48:37.343312+0000","last_undegraded":"2024-09-12T02:48:37.343312+0000","last_fullsized":"2024-09-12T02:48:37.343312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":106,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:21.191836+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:49:21.191836+0000","last_peered":"2024-09-12T02:49:21.191836+0000","last_clean":"2024-09-12T02:49:21.191836+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:49:21.191836+0000","last_undegraded":"2024-09-12T02:49:21.191836+0000","last_fullsized":"2024-09-12T02:49:21.191836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588956+0000","last_change":"2024-09-12T02:48:37.588956+0000","last_active":"2024-09-12T02:48:37.588956+0000","last_peered":"2024-09-12T02:48:37.588956+0000","last_clean":"2024-09-12T02:48:37.588956+0000","last_became_active":"2024-09-12T02:48:37.588561+0000","last_became_peered":"2024-09-12T02:48:37.588561+0000","last_unstale":"2024-09-12T02:48:37.588956+0000","last_undegraded":"2024-09-12T02:48:37.588956+0000","last_fullsized":"2024-09-12T02:48:37.588956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:08.645076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343699+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:48:37.343699+0000","last_peered":"2024-09-12T02:48:37.343699+0000","last_clean":"2024-09-12T02:48:37.343699+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:48:37.343699+0000","last_undegraded":"2024-09-12T02:48:37.343699+0000","last_fullsized":"2024-09-12T02:48:37.343699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340641+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:48:37.340641+0000","last_peered":"2024-09-12T02:48:37.340641+0000","last_clean":"2024-09-12T02:48:37.340641+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:48:37.340641+0000","last_undegraded":"2024-09-12T02:48:37.340641+0000","last_fullsized":"2024-09-12T02:48:37.340641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343329+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:48:37.343329+0000","last_peered":"2024-09-12T02:48:37.343329+0000","last_clean":"2024-09-12T02:48:37.343329+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:48:37.343329+0000","last_undegraded":"2024-09-12T02:48:37.343329+0000","last_fullsized":"2024-09-12T02:48:37.343329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.589056+0000","last_change":"2024-09-12T02:48:37.589056+0000","last_active":"2024-09-12T02:48:37.589056+0000","last_peered":"2024-09-12T02:48:37.589056+0000","last_clean":"2024-09-12T02:48:37.589056+0000","last_became_active":"2024-09-12T02:48:37.588629+0000","last_became_peered":"2024-09-12T02:48:37.588629+0000","last_unstale":"2024-09-12T02:48:37.589056+0000","last_undegraded":"2024-09-12T02:48:37.589056+0000","last_fullsized":"2024-09-12T02:48:37.589056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:03.490341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340249+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:48:37.340249+0000","last_peered":"2024-09-12T02:48:37.340249+0000","last_clean":"2024-09-12T02:48:37.340249+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:48:37.340249+0000","last_undegraded":"2024-09-12T02:48:37.340249+0000","last_fullsized":"2024-09-12T02:48:37.340249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336590+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:48:37.336590+0000","last_peered":"2024-09-12T02:48:37.336590+0000","last_clean":"2024-09-12T02:48:37.336590+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:48:37.336590+0000","last_undegraded":"2024-09-12T02:48:37.336590+0000","last_fullsized":"2024-09-12T02:48:37.336590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341713+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:48:37.341713+0000","last_peered":"2024-09-12T02:48:37.341713+0000","last_clean":"2024-09-12T02:48:37.341713+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:48:37.341713+0000","last_undegraded":"2024-09-12T02:48:37.341713+0000","last_fullsized":"2024-09-12T02:48:37.341713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343987+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:48:37.343987+0000","last_peered":"2024-09-12T02:48:37.343987+0000","last_clean":"2024-09-12T02:48:37.343987+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:48:37.343987+0000","last_undegraded":"2024-09-12T02:48:37.343987+0000","last_fullsized":"2024-09-12T02:48:37.343987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345556+0000","last_change":"2024-09-12T02:48:37.345556+0000","last_active":"2024-09-12T02:48:37.345556+0000","last_peered":"2024-09-12T02:48:37.345556+0000","last_clean":"2024-09-12T02:48:37.345556+0000","last_became_active":"2024-09-12T02:48:37.345220+0000","last_became_peered":"2024-09-12T02:48:37.345220+0000","last_unstale":"2024-09-12T02:48:37.345556+0000","last_undegraded":"2024-09-12T02:48:37.345556+0000","last_fullsized":"2024-09-12T02:48:37.345556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:25.562372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337339+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:37.337339+0000","last_peered":"2024-09-12T02:48:37.337339+0000","last_clean":"2024-09-12T02:48:37.337339+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:37.337339+0000","last_undegraded":"2024-09-12T02:48:37.337339+0000","last_fullsized":"2024-09-12T02:48:37.337339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339282+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:48:37.339282+0000","last_peered":"2024-09-12T02:48:37.339282+0000","last_clean":"2024-09-12T02:48:37.339282+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:48:37.339282+0000","last_undegraded":"2024-09-12T02:48:37.339282+0000","last_fullsized":"2024-09-12T02:48:37.339282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339814+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:48:37.339814+0000","last_peered":"2024-09-12T02:48:37.339814+0000","last_clean":"2024-09-12T02:48:37.339814+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:48:37.339814+0000","last_undegraded":"2024-09-12T02:48:37.339814+0000","last_fullsized":"2024-09-12T02:48:37.339814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336265+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:48:37.336265+0000","last_peered":"2024-09-12T02:48:37.336265+0000","last_clean":"2024-09-12T02:48:37.336265+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:48:37.336265+0000","last_undegraded":"2024-09-12T02:48:37.336265+0000","last_fullsized":"2024-09-12T02:48:37.336265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341263+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:48:37.341263+0000","last_peered":"2024-09-12T02:48:37.341263+0000","last_clean":"2024-09-12T02:48:37.341263+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:48:37.341263+0000","last_undegraded":"2024-09-12T02:48:37.341263+0000","last_fullsized":"2024-09-12T02:48:37.341263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343830+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:48:37.343830+0000","last_peered":"2024-09-12T02:48:37.343830+0000","last_clean":"2024-09-12T02:48:37.343830+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:48:37.343830+0000","last_undegraded":"2024-09-12T02:48:37.343830+0000","last_fullsized":"2024-09-12T02:48:37.343830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588060+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:37.588060+0000","last_peered":"2024-09-12T02:48:37.588060+0000","last_clean":"2024-09-12T02:48:37.588060+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:37.588060+0000","last_undegraded":"2024-09-12T02:48:37.588060+0000","last_fullsized":"2024-09-12T02:48:37.588060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342161+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:48:37.342161+0000","last_peered":"2024-09-12T02:48:37.342161+0000","last_clean":"2024-09-12T02:48:37.342161+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:48:37.342161+0000","last_undegraded":"2024-09-12T02:48:37.342161+0000","last_fullsized":"2024-09-12T02:48:37.342161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339233+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:48:37.339233+0000","last_peered":"2024-09-12T02:48:37.339233+0000","last_clean":"2024-09-12T02:48:37.339233+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:48:37.339233+0000","last_undegraded":"2024-09-12T02:48:37.339233+0000","last_fullsized":"2024-09-12T02:48:37.339233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339694+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:48:37.339694+0000","last_peered":"2024-09-12T02:48:37.339694+0000","last_clean":"2024-09-12T02:48:37.339694+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:48:37.339694+0000","last_undegraded":"2024-09-12T02:48:37.339694+0000","last_fullsized":"2024-09-12T02:48:37.339694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339992+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:48:37.339992+0000","last_peered":"2024-09-12T02:48:37.339992+0000","last_clean":"2024-09-12T02:48:37.339992+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:48:37.339992+0000","last_undegraded":"2024-09-12T02:48:37.339992+0000","last_fullsized":"2024-09-12T02:48:37.339992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336229+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:48:37.336229+0000","last_peered":"2024-09-12T02:48:37.336229+0000","last_clean":"2024-09-12T02:48:37.336229+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:48:37.336229+0000","last_undegraded":"2024-09-12T02:48:37.336229+0000","last_fullsized":"2024-09-12T02:48:37.336229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341015+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:37.341015+0000","last_peered":"2024-09-12T02:48:37.341015+0000","last_clean":"2024-09-12T02:48:37.341015+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:37.341015+0000","last_undegraded":"2024-09-12T02:48:37.341015+0000","last_fullsized":"2024-09-12T02:48:37.341015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587114+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:48:37.587114+0000","last_peered":"2024-09-12T02:48:37.587114+0000","last_clean":"2024-09-12T02:48:37.587114+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:48:37.587114+0000","last_undegraded":"2024-09-12T02:48:37.587114+0000","last_fullsized":"2024-09-12T02:48:37.587114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343668+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:48:37.343668+0000","last_peered":"2024-09-12T02:48:37.343668+0000","last_clean":"2024-09-12T02:48:37.343668+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:48:37.343668+0000","last_undegraded":"2024-09-12T02:48:37.343668+0000","last_fullsized":"2024-09-12T02:48:37.343668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340009+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:48:37.340009+0000","last_peered":"2024-09-12T02:48:37.340009+0000","last_clean":"2024-09-12T02:48:37.340009+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:48:37.340009+0000","last_undegraded":"2024-09-12T02:48:37.340009+0000","last_fullsized":"2024-09-12T02:48:37.340009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339344+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:48:37.339344+0000","last_peered":"2024-09-12T02:48:37.339344+0000","last_clean":"2024-09-12T02:48:37.339344+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:48:37.339344+0000","last_undegraded":"2024-09-12T02:48:37.339344+0000","last_fullsized":"2024-09-12T02:48:37.339344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586757+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:48:37.586757+0000","last_peered":"2024-09-12T02:48:37.586757+0000","last_clean":"2024-09-12T02:48:37.586757+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:48:37.586757+0000","last_undegraded":"2024-09-12T02:48:37.586757+0000","last_fullsized":"2024-09-12T02:48:37.586757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342802+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:48:37.342802+0000","last_peered":"2024-09-12T02:48:37.342802+0000","last_clean":"2024-09-12T02:48:37.342802+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:48:37.342802+0000","last_undegraded":"2024-09-12T02:48:37.342802+0000","last_fullsized":"2024-09-12T02:48:37.342802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339808+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:48:37.339808+0000","last_peered":"2024-09-12T02:48:37.339808+0000","last_clean":"2024-09-12T02:48:37.339808+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:48:37.339808+0000","last_undegraded":"2024-09-12T02:48:37.339808+0000","last_fullsized":"2024-09-12T02:48:37.339808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340891+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:37.340891+0000","last_peered":"2024-09-12T02:48:37.340891+0000","last_clean":"2024-09-12T02:48:37.340891+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:37.340891+0000","last_undegraded":"2024-09-12T02:48:37.340891+0000","last_fullsized":"2024-09-12T02:48:37.340891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340897+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:48:37.340897+0000","last_peered":"2024-09-12T02:48:37.340897+0000","last_clean":"2024-09-12T02:48:37.340897+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:48:37.340897+0000","last_undegraded":"2024-09-12T02:48:37.340897+0000","last_fullsized":"2024-09-12T02:48:37.340897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341040+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:48:37.341040+0000","last_peered":"2024-09-12T02:48:37.341040+0000","last_clean":"2024-09-12T02:48:37.341040+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:48:37.341040+0000","last_undegraded":"2024-09-12T02:48:37.341040+0000","last_fullsized":"2024-09-12T02:48:37.341040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342054+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:48:37.342054+0000","last_peered":"2024-09-12T02:48:37.342054+0000","last_clean":"2024-09-12T02:48:37.342054+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:48:37.342054+0000","last_undegraded":"2024-09-12T02:48:37.342054+0000","last_fullsized":"2024-09-12T02:48:37.342054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":404,"num_read_kb":275,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":501459,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1152757,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906989,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":505757,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":501581,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364855,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":1097492,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":496837,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1056768,"data_stored":1093080,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":501487,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":497031,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":562520,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":560811,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":498225,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:49:27.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:49:27.531 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:27.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:27 smithi046 ceph-mon[30951]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:27.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:27 smithi046 ceph-mon[30951]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:27 smithi196 ceph-mon[34587]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:27 smithi196 ceph-mon[34587]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:27.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:27 smithi145 ceph-mon[34674]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:27.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:27 smithi145 ceph-mon[34674]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 -- 172.21.15.46:0/3258190222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0101de0 msgr2=0x7ff3f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 --2- 172.21.15.46:0/3258190222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0101de0 0x7ff3f01021c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009900 tx=0x7ff3e002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 -- 172.21.15.46:0/3258190222 shutdown_connections 2024-09-12T02:49:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 --2- 172.21.15.46:0/3258190222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 0x7ff3f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 --2- 172.21.15.46:0/3258190222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0102790 0x7ff3f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 --2- 172.21.15.46:0/3258190222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0101de0 0x7ff3f01021c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 -- 172.21.15.46:0/3258190222 >> 172.21.15.46:0/3258190222 conn(0x7ff3f00fd8c0 msgr2=0x7ff3f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 -- 172.21.15.46:0/3258190222 shutdown_connections 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.060+0000 7ff3f6384640 1 -- 172.21.15.46:0/3258190222 wait complete. 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.061+0000 7ff3f6384640 1 Processor -- start 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.061+0000 7ff3f6384640 1 -- start start 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0101de0 0x7ff3f0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 0x7ff3f019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0111e70 con 0x7ff3f0102790 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0111cf0 con 0x7ff3f0106f10 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0111ff0 con 0x7ff3f0101de0 2024-09-12T02:49:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49620/0 (socket says 172.21.15.46:49620) 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 -- 172.21.15.46:0/285480053 learned_addr learned my addr 172.21.15.46:0/285480053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f4921640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 0x7ff3f019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3effff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0101de0 0x7ff3f0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 -- 172.21.15.46:0/285480053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0101de0 msgr2=0x7ff3f0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0101de0 0x7ff3f0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 -- 172.21.15.46:0/285480053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 msgr2=0x7ff3f019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 0x7ff3f019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 -- 172.21.15.46:0/285480053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f019d320 con 0x7ff3f0102790 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3e7fff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c890 tx=0x7ff3d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8004220 con 0x7ff3f0102790 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3d80043c0 con 0x7ff3f0102790 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.062+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f019d610 con 0x7ff3f0102790 2024-09-12T02:49:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.063+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8004a00 con 0x7ff3f0102790 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.063+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01a4ef0 con 0x7ff3f0102790 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.063+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f01021c0 con 0x7ff3f0102790 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.065+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff3d8004ba0 con 0x7ff3f0102790 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.065+0000 7ff3edffb640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 0x7ff3dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.066+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff3d80cf5d0 con 0x7ff3f0102790 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.066+0000 7ff3effff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 0x7ff3dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.066+0000 7ff3effff640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 0x7ff3dc0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.069+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3d8014030 con 0x7ff3f0102790 2024-09-12T02:49:28.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.242+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3f0062630 con 0x7ff3dc0a2c10 2024-09-12T02:49:28.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.251+0000 7ff3edffb640 1 -- 172.21.15.46:0/285480053 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340585 (secure 0 0 0) 0x7ff3f0062630 con 0x7ff3dc0a2c10 2024-09-12T02:49:28.255 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 msgr2=0x7ff3dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 0x7ff3dc0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0031400 comp rx=0 tx=0).stop 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 msgr2=0x7ff3f01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c890 tx=0x7ff3d800cd50 comp rx=0 tx=0).stop 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 shutdown_connections 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3f0106f10 0x7ff3f019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102790 0x7ff3f01988b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff3dc0a2c10 0x7ff3dc0a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 --2- 172.21.15.46:0/285480053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f0101de0 0x7ff3f0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 >> 172.21.15.46:0/285480053 conn(0x7ff3f00fd8c0 msgr2=0x7ff3f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 shutdown_connections 2024-09-12T02:49:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:28.254+0000 7ff3f6384640 1 -- 172.21.15.46:0/285480053 wait complete. 2024-09-12T02:49:29.116 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-12T02:49:26.287041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":540,"num_read_kb":394,"num_write":353,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369644,"kb_used_data":12020,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572180,"statfs":{"total":1151940427776,"available":1151561912320,"internally_reserved":0,"allocated":12308480,"data_stored":7969037,"data_compressed":391518,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003325"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588213+0000","last_change":"2024-09-12T02:47:39.058336+0000","last_active":"2024-09-12T02:48:37.588213+0000","last_peered":"2024-09-12T02:48:37.588213+0000","last_clean":"2024-09-12T02:48:37.588213+0000","last_became_active":"2024-09-12T02:47:39.058000+0000","last_became_peered":"2024-09-12T02:47:39.058000+0000","last_unstale":"2024-09-12T02:48:37.588213+0000","last_undegraded":"2024-09-12T02:48:37.588213+0000","last_fullsized":"2024-09-12T02:48:37.588213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:04.705619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339671+0000","last_change":"2024-09-12T02:47:41.115870+0000","last_active":"2024-09-12T02:48:37.339671+0000","last_peered":"2024-09-12T02:48:37.339671+0000","last_clean":"2024-09-12T02:48:37.339671+0000","last_became_active":"2024-09-12T02:47:41.115551+0000","last_became_peered":"2024-09-12T02:47:41.115551+0000","last_unstale":"2024-09-12T02:48:37.339671+0000","last_undegraded":"2024-09-12T02:48:37.339671+0000","last_fullsized":"2024-09-12T02:48:37.339671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.448846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343048+0000","last_change":"2024-09-12T02:47:48.151849+0000","last_active":"2024-09-12T02:48:37.343048+0000","last_peered":"2024-09-12T02:48:37.343048+0000","last_clean":"2024-09-12T02:48:37.343048+0000","last_became_active":"2024-09-12T02:47:48.151408+0000","last_became_peered":"2024-09-12T02:47:48.151408+0000","last_unstale":"2024-09-12T02:48:37.343048+0000","last_undegraded":"2024-09-12T02:48:37.343048+0000","last_fullsized":"2024-09-12T02:48:37.343048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:33.214904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339882+0000","last_change":"2024-09-12T02:47:39.699428+0000","last_active":"2024-09-12T02:48:37.339882+0000","last_peered":"2024-09-12T02:48:37.339882+0000","last_clean":"2024-09-12T02:48:37.339882+0000","last_became_active":"2024-09-12T02:47:39.699091+0000","last_became_peered":"2024-09-12T02:47:39.699091+0000","last_unstale":"2024-09-12T02:48:37.339882+0000","last_undegraded":"2024-09-12T02:48:37.339882+0000","last_fullsized":"2024-09-12T02:48:37.339882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:17.569147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341105+0000","last_change":"2024-09-12T02:47:41.116284+0000","last_active":"2024-09-12T02:48:37.341105+0000","last_peered":"2024-09-12T02:48:37.341105+0000","last_clean":"2024-09-12T02:48:37.341105+0000","last_became_active":"2024-09-12T02:47:41.115991+0000","last_became_peered":"2024-09-12T02:47:41.115991+0000","last_unstale":"2024-09-12T02:48:37.341105+0000","last_undegraded":"2024-09-12T02:48:37.341105+0000","last_fullsized":"2024-09-12T02:48:37.341105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:01.420384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340351+0000","last_change":"2024-09-12T02:47:48.152538+0000","last_active":"2024-09-12T02:48:37.340351+0000","last_peered":"2024-09-12T02:48:37.340351+0000","last_clean":"2024-09-12T02:48:37.340351+0000","last_became_active":"2024-09-12T02:47:48.152099+0000","last_became_peered":"2024-09-12T02:47:48.152099+0000","last_unstale":"2024-09-12T02:48:37.340351+0000","last_undegraded":"2024-09-12T02:48:37.340351+0000","last_fullsized":"2024-09-12T02:48:37.340351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.176844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341138+0000","last_change":"2024-09-12T02:47:39.064712+0000","last_active":"2024-09-12T02:48:37.341138+0000","last_peered":"2024-09-12T02:48:37.341138+0000","last_clean":"2024-09-12T02:48:37.341138+0000","last_became_active":"2024-09-12T02:47:39.064386+0000","last_became_peered":"2024-09-12T02:47:39.064386+0000","last_unstale":"2024-09-12T02:48:37.341138+0000","last_undegraded":"2024-09-12T02:48:37.341138+0000","last_fullsized":"2024-09-12T02:48:37.341138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:10.010354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339509+0000","last_change":"2024-09-12T02:47:41.117379+0000","last_active":"2024-09-12T02:48:37.339509+0000","last_peered":"2024-09-12T02:48:37.339509+0000","last_clean":"2024-09-12T02:48:37.339509+0000","last_became_active":"2024-09-12T02:47:41.117031+0000","last_became_peered":"2024-09-12T02:47:41.117031+0000","last_unstale":"2024-09-12T02:48:37.339509+0000","last_undegraded":"2024-09-12T02:48:37.339509+0000","last_fullsized":"2024-09-12T02:48:37.339509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:46.468851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341184+0000","last_change":"2024-09-12T02:47:48.151908+0000","last_active":"2024-09-12T02:48:37.341184+0000","last_peered":"2024-09-12T02:48:37.341184+0000","last_clean":"2024-09-12T02:48:37.341184+0000","last_became_active":"2024-09-12T02:47:48.151468+0000","last_became_peered":"2024-09-12T02:47:48.151468+0000","last_unstale":"2024-09-12T02:48:37.341184+0000","last_undegraded":"2024-09-12T02:48:37.341184+0000","last_fullsized":"2024-09-12T02:48:37.341184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:33.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340619+0000","last_change":"2024-09-12T02:47:39.062627+0000","last_active":"2024-09-12T02:48:37.340619+0000","last_peered":"2024-09-12T02:48:37.340619+0000","last_clean":"2024-09-12T02:48:37.340619+0000","last_became_active":"2024-09-12T02:47:39.062258+0000","last_became_peered":"2024-09-12T02:47:39.062258+0000","last_unstale":"2024-09-12T02:48:37.340619+0000","last_undegraded":"2024-09-12T02:48:37.340619+0000","last_fullsized":"2024-09-12T02:48:37.340619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:35.650522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338203+0000","last_change":"2024-09-12T02:47:41.123634+0000","last_active":"2024-09-12T02:48:37.338203+0000","last_peered":"2024-09-12T02:48:37.338203+0000","last_clean":"2024-09-12T02:48:37.338203+0000","last_became_active":"2024-09-12T02:47:41.123233+0000","last_became_peered":"2024-09-12T02:47:41.123233+0000","last_unstale":"2024-09-12T02:48:37.338203+0000","last_undegraded":"2024-09-12T02:48:37.338203+0000","last_fullsized":"2024-09-12T02:48:37.338203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:13.508661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340502+0000","last_change":"2024-09-12T02:47:48.141458+0000","last_active":"2024-09-12T02:48:37.340502+0000","last_peered":"2024-09-12T02:48:37.340502+0000","last_clean":"2024-09-12T02:48:37.340502+0000","last_became_active":"2024-09-12T02:47:48.141024+0000","last_became_peered":"2024-09-12T02:47:48.141024+0000","last_unstale":"2024-09-12T02:48:37.340502+0000","last_undegraded":"2024-09-12T02:48:37.340502+0000","last_fullsized":"2024-09-12T02:48:37.340502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:34.782831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586858+0000","last_change":"2024-09-12T02:47:41.124994+0000","last_active":"2024-09-12T02:48:37.586858+0000","last_peered":"2024-09-12T02:48:37.586858+0000","last_clean":"2024-09-12T02:48:37.586858+0000","last_became_active":"2024-09-12T02:47:41.124695+0000","last_became_peered":"2024-09-12T02:47:41.124695+0000","last_unstale":"2024-09-12T02:48:37.586858+0000","last_undegraded":"2024-09-12T02:48:37.586858+0000","last_fullsized":"2024-09-12T02:48:37.586858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:47.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339937+0000","last_change":"2024-09-12T02:47:39.065570+0000","last_active":"2024-09-12T02:48:37.339937+0000","last_peered":"2024-09-12T02:48:37.339937+0000","last_clean":"2024-09-12T02:48:37.339937+0000","last_became_active":"2024-09-12T02:47:39.065256+0000","last_became_peered":"2024-09-12T02:47:39.065256+0000","last_unstale":"2024-09-12T02:48:37.339937+0000","last_undegraded":"2024-09-12T02:48:37.339937+0000","last_fullsized":"2024-09-12T02:48:37.339937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:19.490232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340671+0000","last_change":"2024-09-12T02:47:48.159119+0000","last_active":"2024-09-12T02:48:37.340671+0000","last_peered":"2024-09-12T02:48:37.340671+0000","last_clean":"2024-09-12T02:48:37.340671+0000","last_became_active":"2024-09-12T02:47:48.158704+0000","last_became_peered":"2024-09-12T02:47:48.158704+0000","last_unstale":"2024-09-12T02:48:37.340671+0000","last_undegraded":"2024-09-12T02:48:37.340671+0000","last_fullsized":"2024-09-12T02:48:37.340671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:04.360495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339210+0000","last_change":"2024-09-12T02:47:41.114894+0000","last_active":"2024-09-12T02:48:37.339210+0000","last_peered":"2024-09-12T02:48:37.339210+0000","last_clean":"2024-09-12T02:48:37.339210+0000","last_became_active":"2024-09-12T02:47:41.114578+0000","last_became_peered":"2024-09-12T02:47:41.114578+0000","last_unstale":"2024-09-12T02:48:37.339210+0000","last_undegraded":"2024-09-12T02:48:37.339210+0000","last_fullsized":"2024-09-12T02:48:37.339210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:56.345045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343983+0000","last_change":"2024-09-12T02:47:39.062837+0000","last_active":"2024-09-12T02:48:37.343983+0000","last_peered":"2024-09-12T02:48:37.343983+0000","last_clean":"2024-09-12T02:48:37.343983+0000","last_became_active":"2024-09-12T02:47:39.062291+0000","last_became_peered":"2024-09-12T02:47:39.062291+0000","last_unstale":"2024-09-12T02:48:37.343983+0000","last_undegraded":"2024-09-12T02:48:37.343983+0000","last_fullsized":"2024-09-12T02:48:37.343983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:18.202051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341063+0000","last_change":"2024-09-12T02:47:48.152189+0000","last_active":"2024-09-12T02:48:37.341063+0000","last_peered":"2024-09-12T02:48:37.341063+0000","last_clean":"2024-09-12T02:48:37.341063+0000","last_became_active":"2024-09-12T02:47:48.151620+0000","last_became_peered":"2024-09-12T02:47:48.151620+0000","last_unstale":"2024-09-12T02:48:37.341063+0000","last_undegraded":"2024-09-12T02:48:37.341063+0000","last_fullsized":"2024-09-12T02:48:37.341063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:48.447456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340770+0000","last_change":"2024-09-12T02:47:41.114659+0000","last_active":"2024-09-12T02:48:37.340770+0000","last_peered":"2024-09-12T02:48:37.340770+0000","last_clean":"2024-09-12T02:48:37.340770+0000","last_became_active":"2024-09-12T02:47:41.114326+0000","last_became_peered":"2024-09-12T02:47:41.114326+0000","last_unstale":"2024-09-12T02:48:37.340770+0000","last_undegraded":"2024-09-12T02:48:37.340770+0000","last_fullsized":"2024-09-12T02:48:37.340770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:39.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339382+0000","last_change":"2024-09-12T02:47:39.060757+0000","last_active":"2024-09-12T02:48:37.339382+0000","last_peered":"2024-09-12T02:48:37.339382+0000","last_clean":"2024-09-12T02:48:37.339382+0000","last_became_active":"2024-09-12T02:47:39.060446+0000","last_became_peered":"2024-09-12T02:47:39.060446+0000","last_unstale":"2024-09-12T02:48:37.339382+0000","last_undegraded":"2024-09-12T02:48:37.339382+0000","last_fullsized":"2024-09-12T02:48:37.339382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:40.278878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588035+0000","last_change":"2024-09-12T02:47:48.154655+0000","last_active":"2024-09-12T02:48:37.588035+0000","last_peered":"2024-09-12T02:48:37.588035+0000","last_clean":"2024-09-12T02:48:37.588035+0000","last_became_active":"2024-09-12T02:47:48.154295+0000","last_became_peered":"2024-09-12T02:47:48.154295+0000","last_unstale":"2024-09-12T02:48:37.588035+0000","last_undegraded":"2024-09-12T02:48:37.588035+0000","last_fullsized":"2024-09-12T02:48:37.588035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:32.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339398+0000","last_change":"2024-09-12T02:47:41.113578+0000","last_active":"2024-09-12T02:48:37.339398+0000","last_peered":"2024-09-12T02:48:37.339398+0000","last_clean":"2024-09-12T02:48:37.339398+0000","last_became_active":"2024-09-12T02:47:41.113178+0000","last_became_peered":"2024-09-12T02:47:41.113178+0000","last_unstale":"2024-09-12T02:48:37.339398+0000","last_undegraded":"2024-09-12T02:48:37.339398+0000","last_fullsized":"2024-09-12T02:48:37.339398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:52.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339366+0000","last_change":"2024-09-12T02:47:39.701793+0000","last_active":"2024-09-12T02:48:37.339366+0000","last_peered":"2024-09-12T02:48:37.339366+0000","last_clean":"2024-09-12T02:48:37.339366+0000","last_became_active":"2024-09-12T02:47:39.701419+0000","last_became_peered":"2024-09-12T02:47:39.701419+0000","last_unstale":"2024-09-12T02:48:37.339366+0000","last_undegraded":"2024-09-12T02:48:37.339366+0000","last_fullsized":"2024-09-12T02:48:37.339366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:26.172978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.590486+0000","last_change":"2024-09-12T02:48:37.590486+0000","last_active":"2024-09-12T02:48:37.590486+0000","last_peered":"2024-09-12T02:48:37.590486+0000","last_clean":"2024-09-12T02:48:37.590486+0000","last_became_active":"2024-09-12T02:48:37.589930+0000","last_became_peered":"2024-09-12T02:48:37.589930+0000","last_unstale":"2024-09-12T02:48:37.590486+0000","last_undegraded":"2024-09-12T02:48:37.590486+0000","last_fullsized":"2024-09-12T02:48:37.590486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:42.323965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341047+0000","last_change":"2024-09-12T02:47:41.123058+0000","last_active":"2024-09-12T02:48:37.341047+0000","last_peered":"2024-09-12T02:48:37.341047+0000","last_clean":"2024-09-12T02:48:37.341047+0000","last_became_active":"2024-09-12T02:47:41.122759+0000","last_became_peered":"2024-09-12T02:47:41.122759+0000","last_unstale":"2024-09-12T02:48:37.341047+0000","last_undegraded":"2024-09-12T02:48:37.341047+0000","last_fullsized":"2024-09-12T02:48:37.341047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:12.963017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340953+0000","last_change":"2024-09-12T02:47:39.148023+0000","last_active":"2024-09-12T02:48:37.340953+0000","last_peered":"2024-09-12T02:48:37.340953+0000","last_clean":"2024-09-12T02:48:37.340953+0000","last_became_active":"2024-09-12T02:47:39.147714+0000","last_became_peered":"2024-09-12T02:47:39.147714+0000","last_unstale":"2024-09-12T02:48:37.340953+0000","last_undegraded":"2024-09-12T02:48:37.340953+0000","last_fullsized":"2024-09-12T02:48:37.340953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:23.263957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343184+0000","last_change":"2024-09-12T02:47:48.146928+0000","last_active":"2024-09-12T02:48:37.343184+0000","last_peered":"2024-09-12T02:48:37.343184+0000","last_clean":"2024-09-12T02:48:37.343184+0000","last_became_active":"2024-09-12T02:47:48.146526+0000","last_became_peered":"2024-09-12T02:47:48.146526+0000","last_unstale":"2024-09-12T02:48:37.343184+0000","last_undegraded":"2024-09-12T02:48:37.343184+0000","last_fullsized":"2024-09-12T02:48:37.343184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:41.528800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339781+0000","last_change":"2024-09-12T02:47:41.128198+0000","last_active":"2024-09-12T02:48:37.339781+0000","last_peered":"2024-09-12T02:48:37.339781+0000","last_clean":"2024-09-12T02:48:37.339781+0000","last_became_active":"2024-09-12T02:47:41.127827+0000","last_became_peered":"2024-09-12T02:47:41.127827+0000","last_unstale":"2024-09-12T02:48:37.339781+0000","last_undegraded":"2024-09-12T02:48:37.339781+0000","last_fullsized":"2024-09-12T02:48:37.339781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:50.954003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588143+0000","last_change":"2024-09-12T02:47:39.055804+0000","last_active":"2024-09-12T02:48:37.588143+0000","last_peered":"2024-09-12T02:48:37.588143+0000","last_clean":"2024-09-12T02:48:37.588143+0000","last_became_active":"2024-09-12T02:47:39.055418+0000","last_became_peered":"2024-09-12T02:47:39.055418+0000","last_unstale":"2024-09-12T02:48:37.588143+0000","last_undegraded":"2024-09-12T02:48:37.588143+0000","last_fullsized":"2024-09-12T02:48:37.588143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:54.729949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339449+0000","last_change":"2024-09-12T02:47:48.158006+0000","last_active":"2024-09-12T02:48:37.339449+0000","last_peered":"2024-09-12T02:48:37.339449+0000","last_clean":"2024-09-12T02:48:37.339449+0000","last_became_active":"2024-09-12T02:47:48.157573+0000","last_became_peered":"2024-09-12T02:47:48.157573+0000","last_unstale":"2024-09-12T02:48:37.339449+0000","last_undegraded":"2024-09-12T02:48:37.339449+0000","last_fullsized":"2024-09-12T02:48:37.339449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:09.423353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336705+0000","last_change":"2024-09-12T02:47:41.119672+0000","last_active":"2024-09-12T02:48:37.336705+0000","last_peered":"2024-09-12T02:48:37.336705+0000","last_clean":"2024-09-12T02:48:37.336705+0000","last_became_active":"2024-09-12T02:47:41.119365+0000","last_became_peered":"2024-09-12T02:47:41.119365+0000","last_unstale":"2024-09-12T02:48:37.336705+0000","last_undegraded":"2024-09-12T02:48:37.336705+0000","last_fullsized":"2024-09-12T02:48:37.336705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:04.423298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341250+0000","last_change":"2024-09-12T02:47:39.701876+0000","last_active":"2024-09-12T02:48:37.341250+0000","last_peered":"2024-09-12T02:48:37.341250+0000","last_clean":"2024-09-12T02:48:37.341250+0000","last_became_active":"2024-09-12T02:47:39.701427+0000","last_became_peered":"2024-09-12T02:47:39.701427+0000","last_unstale":"2024-09-12T02:48:37.341250+0000","last_undegraded":"2024-09-12T02:48:37.341250+0000","last_fullsized":"2024-09-12T02:48:37.341250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:35.533499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341552+0000","last_change":"2024-09-12T02:47:48.146986+0000","last_active":"2024-09-12T02:48:37.341552+0000","last_peered":"2024-09-12T02:48:37.341552+0000","last_clean":"2024-09-12T02:48:37.341552+0000","last_became_active":"2024-09-12T02:47:48.146547+0000","last_became_peered":"2024-09-12T02:47:48.146547+0000","last_unstale":"2024-09-12T02:48:37.341552+0000","last_undegraded":"2024-09-12T02:48:37.341552+0000","last_fullsized":"2024-09-12T02:48:37.341552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:52.543848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343136+0000","last_change":"2024-09-12T02:47:41.110423+0000","last_active":"2024-09-12T02:48:37.343136+0000","last_peered":"2024-09-12T02:48:37.343136+0000","last_clean":"2024-09-12T02:48:37.343136+0000","last_became_active":"2024-09-12T02:47:41.110098+0000","last_became_peered":"2024-09-12T02:47:41.110098+0000","last_unstale":"2024-09-12T02:48:37.343136+0000","last_undegraded":"2024-09-12T02:48:37.343136+0000","last_fullsized":"2024-09-12T02:48:37.343136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:05.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587950+0000","last_change":"2024-09-12T02:47:39.700239+0000","last_active":"2024-09-12T02:48:37.587950+0000","last_peered":"2024-09-12T02:48:37.587950+0000","last_clean":"2024-09-12T02:48:37.587950+0000","last_became_active":"2024-09-12T02:47:39.699912+0000","last_became_peered":"2024-09-12T02:47:39.699912+0000","last_unstale":"2024-09-12T02:48:37.587950+0000","last_undegraded":"2024-09-12T02:48:37.587950+0000","last_fullsized":"2024-09-12T02:48:37.587950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:03.375297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339541+0000","last_change":"2024-09-12T02:47:48.158946+0000","last_active":"2024-09-12T02:48:37.339541+0000","last_peered":"2024-09-12T02:48:37.339541+0000","last_clean":"2024-09-12T02:48:37.339541+0000","last_became_active":"2024-09-12T02:47:48.158533+0000","last_became_peered":"2024-09-12T02:47:48.158533+0000","last_unstale":"2024-09-12T02:48:37.339541+0000","last_undegraded":"2024-09-12T02:48:37.339541+0000","last_fullsized":"2024-09-12T02:48:37.339541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:07.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588702+0000","last_change":"2024-09-12T02:48:37.588702+0000","last_active":"2024-09-12T02:48:37.588702+0000","last_peered":"2024-09-12T02:48:37.588702+0000","last_clean":"2024-09-12T02:48:37.588702+0000","last_became_active":"2024-09-12T02:48:37.588347+0000","last_became_peered":"2024-09-12T02:48:37.588347+0000","last_unstale":"2024-09-12T02:48:37.588702+0000","last_undegraded":"2024-09-12T02:48:37.588702+0000","last_fullsized":"2024-09-12T02:48:37.588702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:30.609070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586990+0000","last_change":"2024-09-12T02:47:39.061841+0000","last_active":"2024-09-12T02:48:37.586990+0000","last_peered":"2024-09-12T02:48:37.586990+0000","last_clean":"2024-09-12T02:48:37.586990+0000","last_became_active":"2024-09-12T02:47:39.061367+0000","last_became_peered":"2024-09-12T02:47:39.061367+0000","last_unstale":"2024-09-12T02:48:37.586990+0000","last_undegraded":"2024-09-12T02:48:37.586990+0000","last_fullsized":"2024-09-12T02:48:37.586990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:28.476663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336464+0000","last_change":"2024-09-12T02:47:48.146751+0000","last_active":"2024-09-12T02:48:37.336464+0000","last_peered":"2024-09-12T02:48:37.336464+0000","last_clean":"2024-09-12T02:48:37.336464+0000","last_became_active":"2024-09-12T02:47:48.146360+0000","last_became_peered":"2024-09-12T02:47:48.146360+0000","last_unstale":"2024-09-12T02:48:37.336464+0000","last_undegraded":"2024-09-12T02:48:37.336464+0000","last_fullsized":"2024-09-12T02:48:37.336464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:28.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337504+0000","last_change":"2024-09-12T02:47:41.125885+0000","last_active":"2024-09-12T02:48:37.337504+0000","last_peered":"2024-09-12T02:48:37.337504+0000","last_clean":"2024-09-12T02:48:37.337504+0000","last_became_active":"2024-09-12T02:47:41.125579+0000","last_became_peered":"2024-09-12T02:47:41.125579+0000","last_unstale":"2024-09-12T02:48:37.337504+0000","last_undegraded":"2024-09-12T02:48:37.337504+0000","last_fullsized":"2024-09-12T02:48:37.337504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:57.483323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:23.194259+0000","last_change":"2024-09-12T02:47:39.699342+0000","last_active":"2024-09-12T02:49:23.194259+0000","last_peered":"2024-09-12T02:49:23.194259+0000","last_clean":"2024-09-12T02:49:23.194259+0000","last_became_active":"2024-09-12T02:47:39.698986+0000","last_became_peered":"2024-09-12T02:47:39.698986+0000","last_unstale":"2024-09-12T02:49:23.194259+0000","last_undegraded":"2024-09-12T02:49:23.194259+0000","last_fullsized":"2024-09-12T02:49:23.194259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.286921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336272+0000","last_change":"2024-09-12T02:47:48.159646+0000","last_active":"2024-09-12T02:48:37.336272+0000","last_peered":"2024-09-12T02:48:37.336272+0000","last_clean":"2024-09-12T02:48:37.336272+0000","last_became_active":"2024-09-12T02:47:48.158681+0000","last_became_peered":"2024-09-12T02:47:48.158681+0000","last_unstale":"2024-09-12T02:48:37.336272+0000","last_undegraded":"2024-09-12T02:48:37.336272+0000","last_fullsized":"2024-09-12T02:48:37.336272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:55.330523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588802+0000","last_change":"2024-09-12T02:48:37.588802+0000","last_active":"2024-09-12T02:48:37.588802+0000","last_peered":"2024-09-12T02:48:37.588802+0000","last_clean":"2024-09-12T02:48:37.588802+0000","last_became_active":"2024-09-12T02:48:37.588326+0000","last_became_peered":"2024-09-12T02:48:37.588326+0000","last_unstale":"2024-09-12T02:48:37.588802+0000","last_undegraded":"2024-09-12T02:48:37.588802+0000","last_fullsized":"2024-09-12T02:48:37.588802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:32.340488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586758+0000","last_change":"2024-09-12T02:47:39.063187+0000","last_active":"2024-09-12T02:48:37.586758+0000","last_peered":"2024-09-12T02:48:37.586758+0000","last_clean":"2024-09-12T02:48:37.586758+0000","last_became_active":"2024-09-12T02:47:39.062883+0000","last_became_peered":"2024-09-12T02:47:39.062883+0000","last_unstale":"2024-09-12T02:48:37.586758+0000","last_undegraded":"2024-09-12T02:48:37.586758+0000","last_fullsized":"2024-09-12T02:48:37.586758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:09.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345128+0000","last_change":"2024-09-12T02:48:37.345128+0000","last_active":"2024-09-12T02:48:37.345128+0000","last_peered":"2024-09-12T02:48:37.345128+0000","last_clean":"2024-09-12T02:48:37.345128+0000","last_became_active":"2024-09-12T02:48:37.344670+0000","last_became_peered":"2024-09-12T02:48:37.344670+0000","last_unstale":"2024-09-12T02:48:37.345128+0000","last_undegraded":"2024-09-12T02:48:37.345128+0000","last_fullsized":"2024-09-12T02:48:37.345128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:32.206800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6,10],"acting":[5,8,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339329+0000","last_change":"2024-09-12T02:47:41.116092+0000","last_active":"2024-09-12T02:48:37.339329+0000","last_peered":"2024-09-12T02:48:37.339329+0000","last_clean":"2024-09-12T02:48:37.339329+0000","last_became_active":"2024-09-12T02:47:41.115046+0000","last_became_peered":"2024-09-12T02:47:41.115046+0000","last_unstale":"2024-09-12T02:48:37.339329+0000","last_undegraded":"2024-09-12T02:48:37.339329+0000","last_fullsized":"2024-09-12T02:48:37.339329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:01.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341367+0000","last_change":"2024-09-12T02:47:39.061596+0000","last_active":"2024-09-12T02:48:37.341367+0000","last_peered":"2024-09-12T02:48:37.341367+0000","last_clean":"2024-09-12T02:48:37.341367+0000","last_became_active":"2024-09-12T02:47:39.061259+0000","last_became_peered":"2024-09-12T02:47:39.061259+0000","last_unstale":"2024-09-12T02:48:37.341367+0000","last_undegraded":"2024-09-12T02:48:37.341367+0000","last_fullsized":"2024-09-12T02:48:37.341367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:32.864468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586962+0000","last_change":"2024-09-12T02:46:24.614814+0000","last_active":"2024-09-12T02:48:37.586962+0000","last_peered":"2024-09-12T02:48:37.586962+0000","last_clean":"2024-09-12T02:48:37.586962+0000","last_became_active":"2024-09-12T02:46:24.614367+0000","last_became_peered":"2024-09-12T02:46:24.614367+0000","last_unstale":"2024-09-12T02:48:37.586962+0000","last_undegraded":"2024-09-12T02:48:37.586962+0000","last_fullsized":"2024-09-12T02:48:37.586962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:42:56.922890+0000","last_clean_scrub_stamp":"2024-09-12T02:42:56.922890+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:08:27.895799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586973+0000","last_change":"2024-09-12T02:47:48.155490+0000","last_active":"2024-09-12T02:48:37.586973+0000","last_peered":"2024-09-12T02:48:37.586973+0000","last_clean":"2024-09-12T02:48:37.586973+0000","last_became_active":"2024-09-12T02:47:48.155067+0000","last_became_peered":"2024-09-12T02:47:48.155067+0000","last_unstale":"2024-09-12T02:48:37.586973+0000","last_undegraded":"2024-09-12T02:48:37.586973+0000","last_fullsized":"2024-09-12T02:48:37.586973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:14.908849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587180+0000","last_change":"2024-09-12T02:47:41.127322+0000","last_active":"2024-09-12T02:48:37.587180+0000","last_peered":"2024-09-12T02:48:37.587180+0000","last_clean":"2024-09-12T02:48:37.587180+0000","last_became_active":"2024-09-12T02:47:41.126906+0000","last_became_peered":"2024-09-12T02:47:41.126906+0000","last_unstale":"2024-09-12T02:48:37.587180+0000","last_undegraded":"2024-09-12T02:48:37.587180+0000","last_fullsized":"2024-09-12T02:48:37.587180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:29.612631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343649+0000","last_change":"2024-09-12T02:47:39.699468+0000","last_active":"2024-09-12T02:48:37.343649+0000","last_peered":"2024-09-12T02:48:37.343649+0000","last_clean":"2024-09-12T02:48:37.343649+0000","last_became_active":"2024-09-12T02:47:39.699057+0000","last_became_peered":"2024-09-12T02:47:39.699057+0000","last_unstale":"2024-09-12T02:48:37.343649+0000","last_undegraded":"2024-09-12T02:48:37.343649+0000","last_fullsized":"2024-09-12T02:48:37.343649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:14.993341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339501+0000","last_change":"2024-09-12T02:47:48.146094+0000","last_active":"2024-09-12T02:48:37.339501+0000","last_peered":"2024-09-12T02:48:37.339501+0000","last_clean":"2024-09-12T02:48:37.339501+0000","last_became_active":"2024-09-12T02:47:48.145480+0000","last_became_peered":"2024-09-12T02:47:48.145480+0000","last_unstale":"2024-09-12T02:48:37.339501+0000","last_undegraded":"2024-09-12T02:48:37.339501+0000","last_fullsized":"2024-09-12T02:48:37.339501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:52.678526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587059+0000","last_change":"2024-09-12T02:47:41.127672+0000","last_active":"2024-09-12T02:48:37.587059+0000","last_peered":"2024-09-12T02:48:37.587059+0000","last_clean":"2024-09-12T02:48:37.587059+0000","last_became_active":"2024-09-12T02:47:41.126889+0000","last_became_peered":"2024-09-12T02:47:41.126889+0000","last_unstale":"2024-09-12T02:48:37.587059+0000","last_undegraded":"2024-09-12T02:48:37.587059+0000","last_fullsized":"2024-09-12T02:48:37.587059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:21.913429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338318+0000","last_change":"2024-09-12T02:47:39.063141+0000","last_active":"2024-09-12T02:48:37.338318+0000","last_peered":"2024-09-12T02:48:37.338318+0000","last_clean":"2024-09-12T02:48:37.338318+0000","last_became_active":"2024-09-12T02:47:39.062802+0000","last_became_peered":"2024-09-12T02:47:39.062802+0000","last_unstale":"2024-09-12T02:48:37.338318+0000","last_undegraded":"2024-09-12T02:48:37.338318+0000","last_fullsized":"2024-09-12T02:48:37.338318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:14.511248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.346098+0000","last_change":"2024-09-12T02:48:37.346098+0000","last_active":"2024-09-12T02:48:37.346098+0000","last_peered":"2024-09-12T02:48:37.346098+0000","last_clean":"2024-09-12T02:48:37.346098+0000","last_became_active":"2024-09-12T02:48:37.345696+0000","last_became_peered":"2024-09-12T02:48:37.345696+0000","last_unstale":"2024-09-12T02:48:37.346098+0000","last_undegraded":"2024-09-12T02:48:37.346098+0000","last_fullsized":"2024-09-12T02:48:37.346098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:48.040441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587347+0000","last_change":"2024-09-12T02:47:41.126052+0000","last_active":"2024-09-12T02:48:37.587347+0000","last_peered":"2024-09-12T02:48:37.587347+0000","last_clean":"2024-09-12T02:48:37.587347+0000","last_became_active":"2024-09-12T02:47:41.125730+0000","last_became_peered":"2024-09-12T02:47:41.125730+0000","last_unstale":"2024-09-12T02:48:37.587347+0000","last_undegraded":"2024-09-12T02:48:37.587347+0000","last_fullsized":"2024-09-12T02:48:37.587347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:29.369688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336481+0000","last_change":"2024-09-12T02:47:39.063247+0000","last_active":"2024-09-12T02:48:37.336481+0000","last_peered":"2024-09-12T02:48:37.336481+0000","last_clean":"2024-09-12T02:48:37.336481+0000","last_became_active":"2024-09-12T02:47:39.062873+0000","last_became_peered":"2024-09-12T02:47:39.062873+0000","last_unstale":"2024-09-12T02:48:37.336481+0000","last_undegraded":"2024-09-12T02:48:37.336481+0000","last_fullsized":"2024-09-12T02:48:37.336481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:22.769923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337296+0000","last_change":"2024-09-12T02:47:48.148003+0000","last_active":"2024-09-12T02:48:37.337296+0000","last_peered":"2024-09-12T02:48:37.337296+0000","last_clean":"2024-09-12T02:48:37.337296+0000","last_became_active":"2024-09-12T02:47:48.147571+0000","last_became_peered":"2024-09-12T02:47:48.147571+0000","last_unstale":"2024-09-12T02:48:37.337296+0000","last_undegraded":"2024-09-12T02:48:37.337296+0000","last_fullsized":"2024-09-12T02:48:37.337296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:49.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341055+0000","last_change":"2024-09-12T02:47:41.119998+0000","last_active":"2024-09-12T02:48:37.341055+0000","last_peered":"2024-09-12T02:48:37.341055+0000","last_clean":"2024-09-12T02:48:37.341055+0000","last_became_active":"2024-09-12T02:47:41.118338+0000","last_became_peered":"2024-09-12T02:47:41.118338+0000","last_unstale":"2024-09-12T02:48:37.341055+0000","last_undegraded":"2024-09-12T02:48:37.341055+0000","last_fullsized":"2024-09-12T02:48:37.341055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:20.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340312+0000","last_change":"2024-09-12T02:47:39.060756+0000","last_active":"2024-09-12T02:48:37.340312+0000","last_peered":"2024-09-12T02:48:37.340312+0000","last_clean":"2024-09-12T02:48:37.340312+0000","last_became_active":"2024-09-12T02:47:39.060393+0000","last_became_peered":"2024-09-12T02:47:39.060393+0000","last_unstale":"2024-09-12T02:48:37.340312+0000","last_undegraded":"2024-09-12T02:48:37.340312+0000","last_fullsized":"2024-09-12T02:48:37.340312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:22.007375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337321+0000","last_change":"2024-09-12T02:47:48.148744+0000","last_active":"2024-09-12T02:48:37.337321+0000","last_peered":"2024-09-12T02:48:37.337321+0000","last_clean":"2024-09-12T02:48:37.337321+0000","last_became_active":"2024-09-12T02:47:48.147980+0000","last_became_peered":"2024-09-12T02:47:48.147980+0000","last_unstale":"2024-09-12T02:48:37.337321+0000","last_undegraded":"2024-09-12T02:48:37.337321+0000","last_fullsized":"2024-09-12T02:48:37.337321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:21.169628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337404+0000","last_change":"2024-09-12T02:47:41.119474+0000","last_active":"2024-09-12T02:48:37.337404+0000","last_peered":"2024-09-12T02:48:37.337404+0000","last_clean":"2024-09-12T02:48:37.337404+0000","last_became_active":"2024-09-12T02:47:41.119128+0000","last_became_peered":"2024-09-12T02:47:41.119128+0000","last_unstale":"2024-09-12T02:48:37.337404+0000","last_undegraded":"2024-09-12T02:48:37.337404+0000","last_fullsized":"2024-09-12T02:48:37.337404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:50.650882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339395+0000","last_change":"2024-09-12T02:47:39.701897+0000","last_active":"2024-09-12T02:48:37.339395+0000","last_peered":"2024-09-12T02:48:37.339395+0000","last_clean":"2024-09-12T02:48:37.339395+0000","last_became_active":"2024-09-12T02:47:39.701490+0000","last_became_peered":"2024-09-12T02:47:39.701490+0000","last_unstale":"2024-09-12T02:48:37.339395+0000","last_undegraded":"2024-09-12T02:48:37.339395+0000","last_fullsized":"2024-09-12T02:48:37.339395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:46.248259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.344145+0000","last_change":"2024-09-12T02:48:37.344145+0000","last_active":"2024-09-12T02:48:37.344145+0000","last_peered":"2024-09-12T02:48:37.344145+0000","last_clean":"2024-09-12T02:48:37.344145+0000","last_became_active":"2024-09-12T02:48:37.343694+0000","last_became_peered":"2024-09-12T02:48:37.343694+0000","last_unstale":"2024-09-12T02:48:37.344145+0000","last_undegraded":"2024-09-12T02:48:37.344145+0000","last_fullsized":"2024-09-12T02:48:37.344145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:55.742228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586644+0000","last_change":"2024-09-12T02:47:41.127396+0000","last_active":"2024-09-12T02:48:37.586644+0000","last_peered":"2024-09-12T02:48:37.586644+0000","last_clean":"2024-09-12T02:48:37.586644+0000","last_became_active":"2024-09-12T02:47:41.127004+0000","last_became_peered":"2024-09-12T02:47:41.127004+0000","last_unstale":"2024-09-12T02:48:37.586644+0000","last_undegraded":"2024-09-12T02:48:37.586644+0000","last_fullsized":"2024-09-12T02:48:37.586644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:54.645127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":101,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:20.190639+0000","last_change":"2024-09-12T02:47:39.064542+0000","last_active":"2024-09-12T02:49:20.190639+0000","last_peered":"2024-09-12T02:49:20.190639+0000","last_clean":"2024-09-12T02:49:20.190639+0000","last_became_active":"2024-09-12T02:47:39.064164+0000","last_became_peered":"2024-09-12T02:47:39.064164+0000","last_unstale":"2024-09-12T02:49:20.190639+0000","last_undegraded":"2024-09-12T02:49:20.190639+0000","last_fullsized":"2024-09-12T02:49:20.190639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:40.891251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587962+0000","last_change":"2024-09-12T02:47:48.156657+0000","last_active":"2024-09-12T02:48:37.587962+0000","last_peered":"2024-09-12T02:48:37.587962+0000","last_clean":"2024-09-12T02:48:37.587962+0000","last_became_active":"2024-09-12T02:47:48.156152+0000","last_became_peered":"2024-09-12T02:47:48.156152+0000","last_unstale":"2024-09-12T02:48:37.587962+0000","last_undegraded":"2024-09-12T02:48:37.587962+0000","last_fullsized":"2024-09-12T02:48:37.587962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:14.157235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587055+0000","last_change":"2024-09-12T02:47:41.124446+0000","last_active":"2024-09-12T02:48:37.587055+0000","last_peered":"2024-09-12T02:48:37.587055+0000","last_clean":"2024-09-12T02:48:37.587055+0000","last_became_active":"2024-09-12T02:47:41.124150+0000","last_became_peered":"2024-09-12T02:47:41.124150+0000","last_unstale":"2024-09-12T02:48:37.587055+0000","last_undegraded":"2024-09-12T02:48:37.587055+0000","last_fullsized":"2024-09-12T02:48:37.587055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:50.415490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337741+0000","last_change":"2024-09-12T02:47:39.060115+0000","last_active":"2024-09-12T02:48:37.337741+0000","last_peered":"2024-09-12T02:48:37.337741+0000","last_clean":"2024-09-12T02:48:37.337741+0000","last_became_active":"2024-09-12T02:47:39.059703+0000","last_became_peered":"2024-09-12T02:47:39.059703+0000","last_unstale":"2024-09-12T02:48:37.337741+0000","last_undegraded":"2024-09-12T02:48:37.337741+0000","last_fullsized":"2024-09-12T02:48:37.337741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:13.439069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341156+0000","last_change":"2024-09-12T02:47:48.151811+0000","last_active":"2024-09-12T02:48:37.341156+0000","last_peered":"2024-09-12T02:48:37.341156+0000","last_clean":"2024-09-12T02:48:37.341156+0000","last_became_active":"2024-09-12T02:47:48.151375+0000","last_became_peered":"2024-09-12T02:47:48.151375+0000","last_unstale":"2024-09-12T02:48:37.341156+0000","last_undegraded":"2024-09-12T02:48:37.341156+0000","last_fullsized":"2024-09-12T02:48:37.341156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:37.081264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336133+0000","last_change":"2024-09-12T02:47:41.129488+0000","last_active":"2024-09-12T02:48:37.336133+0000","last_peered":"2024-09-12T02:48:37.336133+0000","last_clean":"2024-09-12T02:48:37.336133+0000","last_became_active":"2024-09-12T02:47:41.129155+0000","last_became_peered":"2024-09-12T02:47:41.129155+0000","last_unstale":"2024-09-12T02:48:37.336133+0000","last_undegraded":"2024-09-12T02:48:37.336133+0000","last_fullsized":"2024-09-12T02:48:37.336133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:02.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.353056+0000","last_change":"2024-09-12T02:48:37.352674+0000","last_active":"2024-09-12T02:48:37.353056+0000","last_peered":"2024-09-12T02:48:37.353056+0000","last_clean":"2024-09-12T02:48:37.353056+0000","last_became_active":"2024-09-12T02:48:37.345293+0000","last_became_peered":"2024-09-12T02:48:37.345293+0000","last_unstale":"2024-09-12T02:48:37.353056+0000","last_undegraded":"2024-09-12T02:48:37.353056+0000","last_fullsized":"2024-09-12T02:48:37.353056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:00.609598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338121+0000","last_change":"2024-09-12T02:47:48.145191+0000","last_active":"2024-09-12T02:48:37.338121+0000","last_peered":"2024-09-12T02:48:37.338121+0000","last_clean":"2024-09-12T02:48:37.338121+0000","last_became_active":"2024-09-12T02:47:48.144727+0000","last_became_peered":"2024-09-12T02:47:48.144727+0000","last_unstale":"2024-09-12T02:48:37.338121+0000","last_undegraded":"2024-09-12T02:48:37.338121+0000","last_fullsized":"2024-09-12T02:48:37.338121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:03.368986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340161+0000","last_change":"2024-09-12T02:47:41.128327+0000","last_active":"2024-09-12T02:48:37.340161+0000","last_peered":"2024-09-12T02:48:37.340161+0000","last_clean":"2024-09-12T02:48:37.340161+0000","last_became_active":"2024-09-12T02:47:41.127830+0000","last_became_peered":"2024-09-12T02:47:41.127830+0000","last_unstale":"2024-09-12T02:48:37.340161+0000","last_undegraded":"2024-09-12T02:48:37.340161+0000","last_fullsized":"2024-09-12T02:48:37.340161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:05.639169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:21.191278+0000","last_change":"2024-09-12T02:47:39.064602+0000","last_active":"2024-09-12T02:49:21.191278+0000","last_peered":"2024-09-12T02:49:21.191278+0000","last_clean":"2024-09-12T02:49:21.191278+0000","last_became_active":"2024-09-12T02:47:39.064285+0000","last_became_peered":"2024-09-12T02:47:39.064285+0000","last_unstale":"2024-09-12T02:49:21.191278+0000","last_undegraded":"2024-09-12T02:49:21.191278+0000","last_fullsized":"2024-09-12T02:49:21.191278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.061837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":133,"num_read_kb":129,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587651+0000","last_change":"2024-09-12T02:47:48.152908+0000","last_active":"2024-09-12T02:48:37.587651+0000","last_peered":"2024-09-12T02:48:37.587651+0000","last_clean":"2024-09-12T02:48:37.587651+0000","last_became_active":"2024-09-12T02:47:48.152521+0000","last_became_peered":"2024-09-12T02:47:48.152521+0000","last_unstale":"2024-09-12T02:48:37.587651+0000","last_undegraded":"2024-09-12T02:48:37.587651+0000","last_fullsized":"2024-09-12T02:48:37.587651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:22.559508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339610+0000","last_change":"2024-09-12T02:47:41.122055+0000","last_active":"2024-09-12T02:48:37.339610+0000","last_peered":"2024-09-12T02:48:37.339610+0000","last_clean":"2024-09-12T02:48:37.339610+0000","last_became_active":"2024-09-12T02:47:41.121602+0000","last_became_peered":"2024-09-12T02:47:41.121602+0000","last_unstale":"2024-09-12T02:48:37.339610+0000","last_undegraded":"2024-09-12T02:48:37.339610+0000","last_fullsized":"2024-09-12T02:48:37.339610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:05.298960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.338294+0000","last_change":"2024-09-12T02:47:39.699441+0000","last_active":"2024-09-12T02:48:37.338294+0000","last_peered":"2024-09-12T02:48:37.338294+0000","last_clean":"2024-09-12T02:48:37.338294+0000","last_became_active":"2024-09-12T02:47:39.699017+0000","last_became_peered":"2024-09-12T02:47:39.699017+0000","last_unstale":"2024-09-12T02:48:37.338294+0000","last_undegraded":"2024-09-12T02:48:37.338294+0000","last_fullsized":"2024-09-12T02:48:37.338294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:23.580065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341022+0000","last_change":"2024-09-12T02:47:48.151925+0000","last_active":"2024-09-12T02:48:37.341022+0000","last_peered":"2024-09-12T02:48:37.341022+0000","last_clean":"2024-09-12T02:48:37.341022+0000","last_became_active":"2024-09-12T02:47:48.151479+0000","last_became_peered":"2024-09-12T02:47:48.151479+0000","last_unstale":"2024-09-12T02:48:37.341022+0000","last_undegraded":"2024-09-12T02:48:37.341022+0000","last_fullsized":"2024-09-12T02:48:37.341022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.618467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341169+0000","last_change":"2024-09-12T02:47:41.118123+0000","last_active":"2024-09-12T02:48:37.341169+0000","last_peered":"2024-09-12T02:48:37.341169+0000","last_clean":"2024-09-12T02:48:37.341169+0000","last_became_active":"2024-09-12T02:47:41.117785+0000","last_became_peered":"2024-09-12T02:47:41.117785+0000","last_unstale":"2024-09-12T02:48:37.341169+0000","last_undegraded":"2024-09-12T02:48:37.341169+0000","last_fullsized":"2024-09-12T02:48:37.341169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:45.016402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587207+0000","last_change":"2024-09-12T02:47:39.064350+0000","last_active":"2024-09-12T02:48:37.587207+0000","last_peered":"2024-09-12T02:48:37.587207+0000","last_clean":"2024-09-12T02:48:37.587207+0000","last_became_active":"2024-09-12T02:47:39.063994+0000","last_became_peered":"2024-09-12T02:47:39.063994+0000","last_unstale":"2024-09-12T02:48:37.587207+0000","last_undegraded":"2024-09-12T02:48:37.587207+0000","last_fullsized":"2024-09-12T02:48:37.587207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:54.125245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587311+0000","last_change":"2024-09-12T02:47:48.151701+0000","last_active":"2024-09-12T02:48:37.587311+0000","last_peered":"2024-09-12T02:48:37.587311+0000","last_clean":"2024-09-12T02:48:37.587311+0000","last_became_active":"2024-09-12T02:47:48.151323+0000","last_became_peered":"2024-09-12T02:47:48.151323+0000","last_unstale":"2024-09-12T02:48:37.587311+0000","last_undegraded":"2024-09-12T02:48:37.587311+0000","last_fullsized":"2024-09-12T02:48:37.587311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:39.609859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340162+0000","last_change":"2024-09-12T02:47:41.120918+0000","last_active":"2024-09-12T02:48:37.340162+0000","last_peered":"2024-09-12T02:48:37.340162+0000","last_clean":"2024-09-12T02:48:37.340162+0000","last_became_active":"2024-09-12T02:47:41.116256+0000","last_became_peered":"2024-09-12T02:47:41.116256+0000","last_unstale":"2024-09-12T02:48:37.340162+0000","last_undegraded":"2024-09-12T02:48:37.340162+0000","last_fullsized":"2024-09-12T02:48:37.340162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:57.062708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:20.189847+0000","last_change":"2024-09-12T02:47:39.062910+0000","last_active":"2024-09-12T02:49:20.189847+0000","last_peered":"2024-09-12T02:49:20.189847+0000","last_clean":"2024-09-12T02:49:20.189847+0000","last_became_active":"2024-09-12T02:47:39.062539+0000","last_became_peered":"2024-09-12T02:47:39.062539+0000","last_unstale":"2024-09-12T02:49:20.189847+0000","last_undegraded":"2024-09-12T02:49:20.189847+0000","last_fullsized":"2024-09-12T02:49:20.189847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:09.510141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339163+0000","last_change":"2024-09-12T02:47:48.147537+0000","last_active":"2024-09-12T02:48:37.339163+0000","last_peered":"2024-09-12T02:48:37.339163+0000","last_clean":"2024-09-12T02:48:37.339163+0000","last_became_active":"2024-09-12T02:47:48.146917+0000","last_became_peered":"2024-09-12T02:47:48.146917+0000","last_unstale":"2024-09-12T02:48:37.339163+0000","last_undegraded":"2024-09-12T02:48:37.339163+0000","last_fullsized":"2024-09-12T02:48:37.339163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:37.200818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340919+0000","last_change":"2024-09-12T02:47:48.148467+0000","last_active":"2024-09-12T02:48:37.340919+0000","last_peered":"2024-09-12T02:48:37.340919+0000","last_clean":"2024-09-12T02:48:37.340919+0000","last_became_active":"2024-09-12T02:47:48.148061+0000","last_became_peered":"2024-09-12T02:47:48.148061+0000","last_unstale":"2024-09-12T02:48:37.340919+0000","last_undegraded":"2024-09-12T02:48:37.340919+0000","last_fullsized":"2024-09-12T02:48:37.340919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:32.532420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587478+0000","last_change":"2024-09-12T02:47:41.114749+0000","last_active":"2024-09-12T02:48:37.587478+0000","last_peered":"2024-09-12T02:48:37.587478+0000","last_clean":"2024-09-12T02:48:37.587478+0000","last_became_active":"2024-09-12T02:47:41.114439+0000","last_became_peered":"2024-09-12T02:47:41.114439+0000","last_unstale":"2024-09-12T02:48:37.587478+0000","last_undegraded":"2024-09-12T02:48:37.587478+0000","last_fullsized":"2024-09-12T02:48:37.587478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:29.990026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337681+0000","last_change":"2024-09-12T02:47:39.061463+0000","last_active":"2024-09-12T02:48:37.337681+0000","last_peered":"2024-09-12T02:48:37.337681+0000","last_clean":"2024-09-12T02:48:37.337681+0000","last_became_active":"2024-09-12T02:47:39.061036+0000","last_became_peered":"2024-09-12T02:47:39.061036+0000","last_unstale":"2024-09-12T02:48:37.337681+0000","last_undegraded":"2024-09-12T02:48:37.337681+0000","last_fullsized":"2024-09-12T02:48:37.337681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:22.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340797+0000","last_change":"2024-09-12T02:47:48.153795+0000","last_active":"2024-09-12T02:48:37.340797+0000","last_peered":"2024-09-12T02:48:37.340797+0000","last_clean":"2024-09-12T02:48:37.340797+0000","last_became_active":"2024-09-12T02:47:48.153374+0000","last_became_peered":"2024-09-12T02:47:48.153374+0000","last_unstale":"2024-09-12T02:48:37.340797+0000","last_undegraded":"2024-09-12T02:48:37.340797+0000","last_fullsized":"2024-09-12T02:48:37.340797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.047144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339975+0000","last_change":"2024-09-12T02:47:41.117490+0000","last_active":"2024-09-12T02:48:37.339975+0000","last_peered":"2024-09-12T02:48:37.339975+0000","last_clean":"2024-09-12T02:48:37.339975+0000","last_became_active":"2024-09-12T02:47:41.117184+0000","last_became_peered":"2024-09-12T02:47:41.117184+0000","last_unstale":"2024-09-12T02:48:37.339975+0000","last_undegraded":"2024-09-12T02:48:37.339975+0000","last_fullsized":"2024-09-12T02:48:37.339975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:18.204397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336024+0000","last_change":"2024-09-12T02:47:39.058880+0000","last_active":"2024-09-12T02:48:37.336024+0000","last_peered":"2024-09-12T02:48:37.336024+0000","last_clean":"2024-09-12T02:48:37.336024+0000","last_became_active":"2024-09-12T02:47:39.058506+0000","last_became_peered":"2024-09-12T02:47:39.058506+0000","last_unstale":"2024-09-12T02:48:37.336024+0000","last_undegraded":"2024-09-12T02:48:37.336024+0000","last_fullsized":"2024-09-12T02:48:37.336024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:11.127969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343029+0000","last_change":"2024-09-12T02:47:39.059773+0000","last_active":"2024-09-12T02:48:37.343029+0000","last_peered":"2024-09-12T02:48:37.343029+0000","last_clean":"2024-09-12T02:48:37.343029+0000","last_became_active":"2024-09-12T02:47:39.059456+0000","last_became_peered":"2024-09-12T02:47:39.059456+0000","last_unstale":"2024-09-12T02:48:37.343029+0000","last_undegraded":"2024-09-12T02:48:37.343029+0000","last_fullsized":"2024-09-12T02:48:37.343029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:38.482443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337742+0000","last_change":"2024-09-12T02:47:41.124303+0000","last_active":"2024-09-12T02:48:37.337742+0000","last_peered":"2024-09-12T02:48:37.337742+0000","last_clean":"2024-09-12T02:48:37.337742+0000","last_became_active":"2024-09-12T02:47:41.123803+0000","last_became_peered":"2024-09-12T02:47:41.123803+0000","last_unstale":"2024-09-12T02:48:37.337742+0000","last_undegraded":"2024-09-12T02:48:37.337742+0000","last_fullsized":"2024-09-12T02:48:37.337742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:02.756759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343312+0000","last_change":"2024-09-12T02:47:48.151938+0000","last_active":"2024-09-12T02:48:37.343312+0000","last_peered":"2024-09-12T02:48:37.343312+0000","last_clean":"2024-09-12T02:48:37.343312+0000","last_became_active":"2024-09-12T02:47:48.151477+0000","last_became_peered":"2024-09-12T02:47:48.151477+0000","last_unstale":"2024-09-12T02:48:37.343312+0000","last_undegraded":"2024-09-12T02:48:37.343312+0000","last_fullsized":"2024-09-12T02:48:37.343312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:18.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":106,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:49:21.191836+0000","last_change":"2024-09-12T02:47:39.064952+0000","last_active":"2024-09-12T02:49:21.191836+0000","last_peered":"2024-09-12T02:49:21.191836+0000","last_clean":"2024-09-12T02:49:21.191836+0000","last_became_active":"2024-09-12T02:47:39.064571+0000","last_became_peered":"2024-09-12T02:47:39.064571+0000","last_unstale":"2024-09-12T02:49:21.191836+0000","last_undegraded":"2024-09-12T02:49:21.191836+0000","last_fullsized":"2024-09-12T02:49:21.191836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:38.045750+0000","last_clean_scrub_stamp":"2024-09-12T02:47:38.045750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:19.023447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588956+0000","last_change":"2024-09-12T02:48:37.588956+0000","last_active":"2024-09-12T02:48:37.588956+0000","last_peered":"2024-09-12T02:48:37.588956+0000","last_clean":"2024-09-12T02:48:37.588956+0000","last_became_active":"2024-09-12T02:48:37.588561+0000","last_became_peered":"2024-09-12T02:48:37.588561+0000","last_unstale":"2024-09-12T02:48:37.588956+0000","last_undegraded":"2024-09-12T02:48:37.588956+0000","last_fullsized":"2024-09-12T02:48:37.588956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:08.645076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343699+0000","last_change":"2024-09-12T02:47:48.147132+0000","last_active":"2024-09-12T02:48:37.343699+0000","last_peered":"2024-09-12T02:48:37.343699+0000","last_clean":"2024-09-12T02:48:37.343699+0000","last_became_active":"2024-09-12T02:47:48.146748+0000","last_became_peered":"2024-09-12T02:47:48.146748+0000","last_unstale":"2024-09-12T02:48:37.343699+0000","last_undegraded":"2024-09-12T02:48:37.343699+0000","last_fullsized":"2024-09-12T02:48:37.343699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:47.125152+0000","last_clean_scrub_stamp":"2024-09-12T02:47:47.125152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:31.059285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340641+0000","last_change":"2024-09-12T02:47:41.117578+0000","last_active":"2024-09-12T02:48:37.340641+0000","last_peered":"2024-09-12T02:48:37.340641+0000","last_clean":"2024-09-12T02:48:37.340641+0000","last_became_active":"2024-09-12T02:47:41.117230+0000","last_became_peered":"2024-09-12T02:47:41.117230+0000","last_unstale":"2024-09-12T02:48:37.340641+0000","last_undegraded":"2024-09-12T02:48:37.340641+0000","last_fullsized":"2024-09-12T02:48:37.340641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:43.452776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343329+0000","last_change":"2024-09-12T02:47:41.122510+0000","last_active":"2024-09-12T02:48:37.343329+0000","last_peered":"2024-09-12T02:48:37.343329+0000","last_clean":"2024-09-12T02:48:37.343329+0000","last_became_active":"2024-09-12T02:47:41.120812+0000","last_became_peered":"2024-09-12T02:47:41.120812+0000","last_unstale":"2024-09-12T02:48:37.343329+0000","last_undegraded":"2024-09-12T02:48:37.343329+0000","last_fullsized":"2024-09-12T02:48:37.343329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.589056+0000","last_change":"2024-09-12T02:48:37.589056+0000","last_active":"2024-09-12T02:48:37.589056+0000","last_peered":"2024-09-12T02:48:37.589056+0000","last_clean":"2024-09-12T02:48:37.589056+0000","last_became_active":"2024-09-12T02:48:37.588629+0000","last_became_peered":"2024-09-12T02:48:37.588629+0000","last_unstale":"2024-09-12T02:48:37.589056+0000","last_undegraded":"2024-09-12T02:48:37.589056+0000","last_fullsized":"2024-09-12T02:48:37.589056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:03.490341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340249+0000","last_change":"2024-09-12T02:47:41.123131+0000","last_active":"2024-09-12T02:48:37.340249+0000","last_peered":"2024-09-12T02:48:37.340249+0000","last_clean":"2024-09-12T02:48:37.340249+0000","last_became_active":"2024-09-12T02:47:41.122823+0000","last_became_peered":"2024-09-12T02:47:41.122823+0000","last_unstale":"2024-09-12T02:48:37.340249+0000","last_undegraded":"2024-09-12T02:48:37.340249+0000","last_fullsized":"2024-09-12T02:48:37.340249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:36.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336590+0000","last_change":"2024-09-12T02:47:41.118498+0000","last_active":"2024-09-12T02:48:37.336590+0000","last_peered":"2024-09-12T02:48:37.336590+0000","last_clean":"2024-09-12T02:48:37.336590+0000","last_became_active":"2024-09-12T02:47:41.118192+0000","last_became_peered":"2024-09-12T02:47:41.118192+0000","last_unstale":"2024-09-12T02:48:37.336590+0000","last_undegraded":"2024-09-12T02:48:37.336590+0000","last_fullsized":"2024-09-12T02:48:37.336590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:21.656231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341713+0000","last_change":"2024-09-12T02:47:41.118585+0000","last_active":"2024-09-12T02:48:37.341713+0000","last_peered":"2024-09-12T02:48:37.341713+0000","last_clean":"2024-09-12T02:48:37.341713+0000","last_became_active":"2024-09-12T02:47:41.117280+0000","last_became_peered":"2024-09-12T02:47:41.117280+0000","last_unstale":"2024-09-12T02:48:37.341713+0000","last_undegraded":"2024-09-12T02:48:37.341713+0000","last_fullsized":"2024-09-12T02:48:37.341713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.414417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343987+0000","last_change":"2024-09-12T02:47:41.112896+0000","last_active":"2024-09-12T02:48:37.343987+0000","last_peered":"2024-09-12T02:48:37.343987+0000","last_clean":"2024-09-12T02:48:37.343987+0000","last_became_active":"2024-09-12T02:47:41.112570+0000","last_became_peered":"2024-09-12T02:47:41.112570+0000","last_unstale":"2024-09-12T02:48:37.343987+0000","last_undegraded":"2024-09-12T02:48:37.343987+0000","last_fullsized":"2024-09-12T02:48:37.343987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:54.995376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.345556+0000","last_change":"2024-09-12T02:48:37.345556+0000","last_active":"2024-09-12T02:48:37.345556+0000","last_peered":"2024-09-12T02:48:37.345556+0000","last_clean":"2024-09-12T02:48:37.345556+0000","last_became_active":"2024-09-12T02:48:37.345220+0000","last_became_peered":"2024-09-12T02:48:37.345220+0000","last_unstale":"2024-09-12T02:48:37.345556+0000","last_undegraded":"2024-09-12T02:48:37.345556+0000","last_fullsized":"2024-09-12T02:48:37.345556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:25.562372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.337339+0000","last_change":"2024-09-12T02:47:41.125397+0000","last_active":"2024-09-12T02:48:37.337339+0000","last_peered":"2024-09-12T02:48:37.337339+0000","last_clean":"2024-09-12T02:48:37.337339+0000","last_became_active":"2024-09-12T02:47:41.124514+0000","last_became_peered":"2024-09-12T02:47:41.124514+0000","last_unstale":"2024-09-12T02:48:37.337339+0000","last_undegraded":"2024-09-12T02:48:37.337339+0000","last_fullsized":"2024-09-12T02:48:37.337339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:21.920612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339282+0000","last_change":"2024-09-12T02:47:41.114083+0000","last_active":"2024-09-12T02:48:37.339282+0000","last_peered":"2024-09-12T02:48:37.339282+0000","last_clean":"2024-09-12T02:48:37.339282+0000","last_became_active":"2024-09-12T02:47:41.113793+0000","last_became_peered":"2024-09-12T02:47:41.113793+0000","last_unstale":"2024-09-12T02:48:37.339282+0000","last_undegraded":"2024-09-12T02:48:37.339282+0000","last_fullsized":"2024-09-12T02:48:37.339282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:08.024442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339814+0000","last_change":"2024-09-12T02:47:41.127846+0000","last_active":"2024-09-12T02:48:37.339814+0000","last_peered":"2024-09-12T02:48:37.339814+0000","last_clean":"2024-09-12T02:48:37.339814+0000","last_became_active":"2024-09-12T02:47:41.126564+0000","last_became_peered":"2024-09-12T02:47:41.126564+0000","last_unstale":"2024-09-12T02:48:37.339814+0000","last_undegraded":"2024-09-12T02:48:37.339814+0000","last_fullsized":"2024-09-12T02:48:37.339814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:47.509322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336265+0000","last_change":"2024-09-12T02:47:41.129573+0000","last_active":"2024-09-12T02:48:37.336265+0000","last_peered":"2024-09-12T02:48:37.336265+0000","last_clean":"2024-09-12T02:48:37.336265+0000","last_became_active":"2024-09-12T02:47:41.129255+0000","last_became_peered":"2024-09-12T02:47:41.129255+0000","last_unstale":"2024-09-12T02:48:37.336265+0000","last_undegraded":"2024-09-12T02:48:37.336265+0000","last_fullsized":"2024-09-12T02:48:37.336265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:50.768422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341263+0000","last_change":"2024-09-12T02:47:41.117026+0000","last_active":"2024-09-12T02:48:37.341263+0000","last_peered":"2024-09-12T02:48:37.341263+0000","last_clean":"2024-09-12T02:48:37.341263+0000","last_became_active":"2024-09-12T02:47:41.116587+0000","last_became_peered":"2024-09-12T02:47:41.116587+0000","last_unstale":"2024-09-12T02:48:37.341263+0000","last_undegraded":"2024-09-12T02:48:37.341263+0000","last_fullsized":"2024-09-12T02:48:37.341263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:00.400398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343830+0000","last_change":"2024-09-12T02:47:41.115664+0000","last_active":"2024-09-12T02:48:37.343830+0000","last_peered":"2024-09-12T02:48:37.343830+0000","last_clean":"2024-09-12T02:48:37.343830+0000","last_became_active":"2024-09-12T02:47:41.115304+0000","last_became_peered":"2024-09-12T02:47:41.115304+0000","last_unstale":"2024-09-12T02:48:37.343830+0000","last_undegraded":"2024-09-12T02:48:37.343830+0000","last_fullsized":"2024-09-12T02:48:37.343830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:03.021269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.588060+0000","last_change":"2024-09-12T02:47:41.111385+0000","last_active":"2024-09-12T02:48:37.588060+0000","last_peered":"2024-09-12T02:48:37.588060+0000","last_clean":"2024-09-12T02:48:37.588060+0000","last_became_active":"2024-09-12T02:47:41.111159+0000","last_became_peered":"2024-09-12T02:47:41.111159+0000","last_unstale":"2024-09-12T02:48:37.588060+0000","last_undegraded":"2024-09-12T02:48:37.588060+0000","last_fullsized":"2024-09-12T02:48:37.588060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:38.942848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342161+0000","last_change":"2024-09-12T02:47:41.115017+0000","last_active":"2024-09-12T02:48:37.342161+0000","last_peered":"2024-09-12T02:48:37.342161+0000","last_clean":"2024-09-12T02:48:37.342161+0000","last_became_active":"2024-09-12T02:47:41.114244+0000","last_became_peered":"2024-09-12T02:47:41.114244+0000","last_unstale":"2024-09-12T02:48:37.342161+0000","last_undegraded":"2024-09-12T02:48:37.342161+0000","last_fullsized":"2024-09-12T02:48:37.342161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:18.299121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339233+0000","last_change":"2024-09-12T02:47:41.129517+0000","last_active":"2024-09-12T02:48:37.339233+0000","last_peered":"2024-09-12T02:48:37.339233+0000","last_clean":"2024-09-12T02:48:37.339233+0000","last_became_active":"2024-09-12T02:47:41.129188+0000","last_became_peered":"2024-09-12T02:47:41.129188+0000","last_unstale":"2024-09-12T02:48:37.339233+0000","last_undegraded":"2024-09-12T02:48:37.339233+0000","last_fullsized":"2024-09-12T02:48:37.339233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:48.018934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339694+0000","last_change":"2024-09-12T02:47:41.122875+0000","last_active":"2024-09-12T02:48:37.339694+0000","last_peered":"2024-09-12T02:48:37.339694+0000","last_clean":"2024-09-12T02:48:37.339694+0000","last_became_active":"2024-09-12T02:47:41.121704+0000","last_became_peered":"2024-09-12T02:47:41.121704+0000","last_unstale":"2024-09-12T02:48:37.339694+0000","last_undegraded":"2024-09-12T02:48:37.339694+0000","last_fullsized":"2024-09-12T02:48:37.339694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:26.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339992+0000","last_change":"2024-09-12T02:47:41.114696+0000","last_active":"2024-09-12T02:48:37.339992+0000","last_peered":"2024-09-12T02:48:37.339992+0000","last_clean":"2024-09-12T02:48:37.339992+0000","last_became_active":"2024-09-12T02:47:41.114381+0000","last_became_peered":"2024-09-12T02:47:41.114381+0000","last_unstale":"2024-09-12T02:48:37.339992+0000","last_undegraded":"2024-09-12T02:48:37.339992+0000","last_fullsized":"2024-09-12T02:48:37.339992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:50.424211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.336229+0000","last_change":"2024-09-12T02:47:41.115488+0000","last_active":"2024-09-12T02:48:37.336229+0000","last_peered":"2024-09-12T02:48:37.336229+0000","last_clean":"2024-09-12T02:48:37.336229+0000","last_became_active":"2024-09-12T02:47:41.114386+0000","last_became_peered":"2024-09-12T02:47:41.114386+0000","last_unstale":"2024-09-12T02:48:37.336229+0000","last_undegraded":"2024-09-12T02:48:37.336229+0000","last_fullsized":"2024-09-12T02:48:37.336229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:57.027132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341015+0000","last_change":"2024-09-12T02:47:41.119164+0000","last_active":"2024-09-12T02:48:37.341015+0000","last_peered":"2024-09-12T02:48:37.341015+0000","last_clean":"2024-09-12T02:48:37.341015+0000","last_became_active":"2024-09-12T02:47:41.118632+0000","last_became_peered":"2024-09-12T02:47:41.118632+0000","last_unstale":"2024-09-12T02:48:37.341015+0000","last_undegraded":"2024-09-12T02:48:37.341015+0000","last_fullsized":"2024-09-12T02:48:37.341015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:45.288065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.587114+0000","last_change":"2024-09-12T02:47:41.129236+0000","last_active":"2024-09-12T02:48:37.587114+0000","last_peered":"2024-09-12T02:48:37.587114+0000","last_clean":"2024-09-12T02:48:37.587114+0000","last_became_active":"2024-09-12T02:47:41.128866+0000","last_became_peered":"2024-09-12T02:47:41.128866+0000","last_unstale":"2024-09-12T02:48:37.587114+0000","last_undegraded":"2024-09-12T02:48:37.587114+0000","last_fullsized":"2024-09-12T02:48:37.587114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:12.694414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.343668+0000","last_change":"2024-09-12T02:47:41.117710+0000","last_active":"2024-09-12T02:48:37.343668+0000","last_peered":"2024-09-12T02:48:37.343668+0000","last_clean":"2024-09-12T02:48:37.343668+0000","last_became_active":"2024-09-12T02:47:41.117203+0000","last_became_peered":"2024-09-12T02:47:41.117203+0000","last_unstale":"2024-09-12T02:48:37.343668+0000","last_undegraded":"2024-09-12T02:48:37.343668+0000","last_fullsized":"2024-09-12T02:48:37.343668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:12.573718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340009+0000","last_change":"2024-09-12T02:47:41.121944+0000","last_active":"2024-09-12T02:48:37.340009+0000","last_peered":"2024-09-12T02:48:37.340009+0000","last_clean":"2024-09-12T02:48:37.340009+0000","last_became_active":"2024-09-12T02:47:41.121527+0000","last_became_peered":"2024-09-12T02:47:41.121527+0000","last_unstale":"2024-09-12T02:48:37.340009+0000","last_undegraded":"2024-09-12T02:48:37.340009+0000","last_fullsized":"2024-09-12T02:48:37.340009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:46.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339344+0000","last_change":"2024-09-12T02:47:41.117177+0000","last_active":"2024-09-12T02:48:37.339344+0000","last_peered":"2024-09-12T02:48:37.339344+0000","last_clean":"2024-09-12T02:48:37.339344+0000","last_became_active":"2024-09-12T02:47:41.116770+0000","last_became_peered":"2024-09-12T02:47:41.116770+0000","last_unstale":"2024-09-12T02:48:37.339344+0000","last_undegraded":"2024-09-12T02:48:37.339344+0000","last_fullsized":"2024-09-12T02:48:37.339344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:02.999199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.586757+0000","last_change":"2024-09-12T02:47:41.127521+0000","last_active":"2024-09-12T02:48:37.586757+0000","last_peered":"2024-09-12T02:48:37.586757+0000","last_clean":"2024-09-12T02:48:37.586757+0000","last_became_active":"2024-09-12T02:47:41.127140+0000","last_became_peered":"2024-09-12T02:47:41.127140+0000","last_unstale":"2024-09-12T02:48:37.586757+0000","last_undegraded":"2024-09-12T02:48:37.586757+0000","last_fullsized":"2024-09-12T02:48:37.586757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:40.748246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342802+0000","last_change":"2024-09-12T02:47:41.116409+0000","last_active":"2024-09-12T02:48:37.342802+0000","last_peered":"2024-09-12T02:48:37.342802+0000","last_clean":"2024-09-12T02:48:37.342802+0000","last_became_active":"2024-09-12T02:47:41.115186+0000","last_became_peered":"2024-09-12T02:47:41.115186+0000","last_unstale":"2024-09-12T02:48:37.342802+0000","last_undegraded":"2024-09-12T02:48:37.342802+0000","last_fullsized":"2024-09-12T02:48:37.342802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:50.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.339808+0000","last_change":"2024-09-12T02:47:41.117617+0000","last_active":"2024-09-12T02:48:37.339808+0000","last_peered":"2024-09-12T02:48:37.339808+0000","last_clean":"2024-09-12T02:48:37.339808+0000","last_became_active":"2024-09-12T02:47:41.117289+0000","last_became_peered":"2024-09-12T02:47:41.117289+0000","last_unstale":"2024-09-12T02:48:37.339808+0000","last_undegraded":"2024-09-12T02:48:37.339808+0000","last_fullsized":"2024-09-12T02:48:37.339808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.501222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340891+0000","last_change":"2024-09-12T02:47:41.120334+0000","last_active":"2024-09-12T02:48:37.340891+0000","last_peered":"2024-09-12T02:48:37.340891+0000","last_clean":"2024-09-12T02:48:37.340891+0000","last_became_active":"2024-09-12T02:47:41.119962+0000","last_became_peered":"2024-09-12T02:47:41.119962+0000","last_unstale":"2024-09-12T02:48:37.340891+0000","last_undegraded":"2024-09-12T02:48:37.340891+0000","last_fullsized":"2024-09-12T02:48:37.340891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:13.395286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.340897+0000","last_change":"2024-09-12T02:47:41.124754+0000","last_active":"2024-09-12T02:48:37.340897+0000","last_peered":"2024-09-12T02:48:37.340897+0000","last_clean":"2024-09-12T02:48:37.340897+0000","last_became_active":"2024-09-12T02:47:41.124407+0000","last_became_peered":"2024-09-12T02:47:41.124407+0000","last_unstale":"2024-09-12T02:48:37.340897+0000","last_undegraded":"2024-09-12T02:48:37.340897+0000","last_fullsized":"2024-09-12T02:48:37.340897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:22.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.341040+0000","last_change":"2024-09-12T02:47:41.122956+0000","last_active":"2024-09-12T02:48:37.341040+0000","last_peered":"2024-09-12T02:48:37.341040+0000","last_clean":"2024-09-12T02:48:37.341040+0000","last_became_active":"2024-09-12T02:47:41.122642+0000","last_became_peered":"2024-09-12T02:47:41.122642+0000","last_unstale":"2024-09-12T02:48:37.341040+0000","last_undegraded":"2024-09-12T02:48:37.341040+0000","last_fullsized":"2024-09-12T02:48:37.341040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:05.704282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T02:48:37.342054+0000","last_change":"2024-09-12T02:47:41.115822+0000","last_active":"2024-09-12T02:48:37.342054+0000","last_peered":"2024-09-12T02:48:37.342054+0000","last_clean":"2024-09-12T02:48:37.342054+0000","last_became_active":"2024-09-12T02:47:41.114663+0000","last_became_peered":"2024-09-12T02:47:41.114663+0000","last_unstale":"2024-09-12T02:48:37.342054+0000","last_undegraded":"2024-09-12T02:48:37.342054+0000","last_fullsized":"2024-09-12T02:48:37.342054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:47:40.087136+0000","last_clean_scrub_stamp":"2024-09-12T02:47:40.087136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:19.560775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":404,"num_read_kb":275,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214196,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":501459,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1152757,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906989,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":505757,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":501581,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364855,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":1097492,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":496837,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1056768,"data_stored":1093080,"data_compressed":59811,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":501487,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":497031,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":562520,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":560811,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":498225,"data_compressed":23565,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34606,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36246,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:49:29.121 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:49:29.121 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T02:49:29.121 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T02:49:29.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T02:49:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:29.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:29 smithi046 ceph-mon[30951]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:29.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:29 smithi046 ceph-mon[30951]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:29 smithi196 ceph-mon[34587]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:29.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:29 smithi196 ceph-mon[34587]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:29.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:29 smithi145 ceph-mon[34674]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:29.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:29 smithi145 ceph-mon[34674]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:29.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- 172.21.15.46:0/4121863828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3208103660 msgr2=0x7f320810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 --2- 172.21.15.46:0/4121863828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3208103660 0x7f320810bb50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009900 tx=0x7f31fc02a480 comp rx=0 tx=0).stop 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- 172.21.15.46:0/4121863828 shutdown_connections 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 --2- 172.21.15.46:0/4121863828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f320810c170 0x7f320810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 --2- 172.21.15.46:0/4121863828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3208103660 0x7f320810bb50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 --2- 172.21.15.46:0/4121863828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208102d40 0x7f3208103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- 172.21.15.46:0/4121863828 >> 172.21.15.46:0/4121863828 conn(0x7f32080fc7a0 msgr2=0x7f32080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- 172.21.15.46:0/4121863828 shutdown_connections 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- 172.21.15.46:0/4121863828 wait complete. 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 Processor -- start 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.902+0000 7f320d594640 1 -- start start 2024-09-12T02:49:29.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3208102d40 0x7f3208198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208103660 0x7f32081987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3208111e20 con 0x7f320810c170 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3208111ca0 con 0x7f3208103660 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f320d594640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3208111fa0 con 0x7f3208102d40 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49644/0 (socket says 172.21.15.46:49644) 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 -- 172.21.15.46:0/3196173960 learned_addr learned my addr 172.21.15.46:0/3196173960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32067fc640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208103660 0x7f32081987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f3206ffd640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3208102d40 0x7f3208198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 -- 172.21.15.46:0/3196173960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3208102d40 msgr2=0x7f3208198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3208102d40 0x7f3208198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 -- 172.21.15.46:0/3196173960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208103660 msgr2=0x7f32081987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208103660 0x7f32081987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 -- 172.21.15.46:0/3196173960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320819d230 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.903+0000 7f32077fe640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f320000cce0 tx=0x7f320000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.904+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320000f450 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.904+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f320000faa0 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.904+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200018b70 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.904+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320819d520 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.904+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081a4e00 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.905+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3208103120 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.906+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3200018d10 con 0x7f320810c170 2024-09-12T02:49:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.907+0000 7f31effff640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 0x7f31f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.907+0000 7f3206ffd640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 0x7f31f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.907+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f32000cf900 con 0x7f320810c170 2024-09-12T02:49:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.907+0000 7f3206ffd640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 0x7f31f00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005e10 tx=0x7f31f8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:29.911+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3200014030 con 0x7f320810c170 2024-09-12T02:49:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.124+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3208106c50 con 0x7f320810c170 2024-09-12T02:49:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.125+0000 7f31effff640 1 -- 172.21.15.46:0/3196173960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f320009c3a0 con 0x7f320810c170 2024-09-12T02:49:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:30.125 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T02:49:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 msgr2=0x7f31f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 0x7f31f00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005e10 tx=0x7f31f8005da0 comp rx=0 tx=0).stop 2024-09-12T02:49:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 msgr2=0x7f320819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f320000cce0 tx=0x7f320000a620 comp rx=0 tx=0).stop 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 shutdown_connections 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f320810c170 0x7f320819cb50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3208103660 0x7f32081987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f31f00a2c10 0x7f31f00a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 --2- 172.21.15.46:0/3196173960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3208102d40 0x7f3208198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 >> 172.21.15.46:0/3196173960 conn(0x7f32080fc7a0 msgr2=0x7f32080fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 shutdown_connections 2024-09-12T02:49:30.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:30.127+0000 7f320d594640 1 -- 172.21.15.46:0/3196173960 wait complete. 2024-09-12T02:49:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:30 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3196173960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:49:30.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:30 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3196173960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:49:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:30 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3196173960' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:49:30.954 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T02:49:30.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:49:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:31.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:31 smithi046 ceph-mon[30951]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:31 smithi196 ceph-mon[34587]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 -- 172.21.15.46:0/319935800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 msgr2=0x7f95281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 --2- 172.21.15.46:0/319935800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f95281067e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c02a410 comp rx=0 tx=0).stop 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 -- 172.21.15.46:0/319935800 shutdown_connections 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 --2- 172.21.15.46:0/319935800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 0x7f952810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 --2- 172.21.15.46:0/319935800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f95281067e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 --2- 172.21.15.46:0/319935800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f95281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 -- 172.21.15.46:0/319935800 >> 172.21.15.46:0/319935800 conn(0x7f95280fd8c0 msgr2=0x7f95280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 -- 172.21.15.46:0/319935800 shutdown_connections 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.736+0000 7f952e09d640 1 -- 172.21.15.46:0/319935800 wait complete. 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 Processor -- start 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 -- start start 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f952819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 0x7f952819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111e20 con 0x7f9528102790 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111ca0 con 0x7f9528106f10 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f952e09d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111fa0 con 0x7f9528101de0 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f9526ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.737+0000 7f9526ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49664/0 (socket says 172.21.15.46:49664) 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 -- 172.21.15.46:0/2918784141 learned_addr learned my addr 172.21.15.46:0/2918784141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9527fff640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 0x7f952819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f95277fe640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f952819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 -- 172.21.15.46:0/2918784141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 msgr2=0x7f952819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f952819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 -- 172.21.15.46:0/2918784141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 msgr2=0x7f952819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 0x7f952819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 -- 172.21.15.46:0/2918784141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819f550 con 0x7f9528102790 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f95277fe640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f952819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9526ffd640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f951c005940 tx=0x7f951c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03f030 con 0x7f9528102790 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952819f7e0 con 0x7f9528102790 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f951c003aa0 con 0x7f9528102790 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c035070 con 0x7f9528102790 2024-09-12T02:49:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.738+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a7120 con 0x7f9528102790 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.739+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95281032c0 con 0x7f9528102790 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.741+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f951c042030 con 0x7f9528102790 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.742+0000 7f9524ff9640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.742+0000 7f95277fe640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.742+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f951c0effa0 con 0x7f9528102790 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.742+0000 7f95277fe640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 0x7f95100a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f951800abe0 tx=0x7f951800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.745+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f951c0bc9e0 con 0x7f9528102790 2024-09-12T02:49:31.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:31 smithi145 ceph-mon[34674]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.955+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95281021c0 con 0x7f9528102790 2024-09-12T02:49:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.956+0000 7f9524ff9640 1 -- 172.21.15.46:0/2918784141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7f95281021c0 con 0x7f9528102790 2024-09-12T02:49:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:31.957 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:49:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 msgr2=0x7f95100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 0x7f95100a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f951800abe0 tx=0x7f951800a5c0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 msgr2=0x7f952819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f951c005940 tx=0x7f951c004270 comp rx=0 tx=0).stop 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 shutdown_connections 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9528106f10 0x7f952819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9528102790 0x7f952819aae0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f95100a2c10 0x7f95100a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 --2- 172.21.15.46:0/2918784141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9528101de0 0x7f952819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 >> 172.21.15.46:0/2918784141 conn(0x7f95280fd8c0 msgr2=0x7f95280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 shutdown_connections 2024-09-12T02:49:31.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:31.958+0000 7f952e09d640 1 -- 172.21.15.46:0/2918784141 wait complete. 2024-09-12T02:49:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:32 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2918784141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:32.753 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:32 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2918784141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:32.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:49:32.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:32 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2918784141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:33.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:33 smithi046 ceph-mon[30951]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 -- 172.21.15.46:0/2890543380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964101de0 msgr2=0x7f39641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 --2- 172.21.15.46:0/2890543380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964101de0 0x7f39641021c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 -- 172.21.15.46:0/2890543380 shutdown_connections 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 --2- 172.21.15.46:0/2890543380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964106f10 0x7f396410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 --2- 172.21.15.46:0/2890543380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964102790 0x7f39641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 --2- 172.21.15.46:0/2890543380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964101de0 0x7f39641021c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 -- 172.21.15.46:0/2890543380 >> 172.21.15.46:0/2890543380 conn(0x7f39640fd8c0 msgr2=0x7f39640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 -- 172.21.15.46:0/2890543380 shutdown_connections 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.555+0000 7f396c54e640 1 -- 172.21.15.46:0/2890543380 wait complete. 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 Processor -- start 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 -- start start 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 0x7f39641982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964106f10 0x7f396419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3964111de0 con 0x7f3964102790 2024-09-12T02:49:33.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3964111c60 con 0x7f3964101de0 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396c54e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3964111f60 con 0x7f3964106f10 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49680/0 (socket says 172.21.15.46:49680) 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 -- 172.21.15.46:0/3643859698 learned_addr learned my addr 172.21.15.46:0/3643859698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396aaeb640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964106f10 0x7f396419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396a2ea640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 0x7f39641982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 -- 172.21.15.46:0/3643859698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964106f10 msgr2=0x7f396419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964106f10 0x7f396419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 -- 172.21.15.46:0/3643859698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 msgr2=0x7f39641982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 0x7f39641982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f3969ae9640 1 -- 172.21.15.46:0/3643859698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396419d2a0 con 0x7f3964102790 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.556+0000 7f396a2ea640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 0x7f39641982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:33.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f3969ae9640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f395800cce0 tx=0x7f395800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395800f650 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396419d590 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f395800fca0 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958018b90 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.557+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641a4e70 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.558+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39641021c0 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.559+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f395800f7f0 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.560+0000 7f39637fe640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 0x7f394c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.560+0000 7f396a2ea640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 0x7f394c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.560+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f39580cfa10 con 0x7f3964102790 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.561+0000 7f396a2ea640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 0x7f394c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3954031440 tx=0x7f39540095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:33.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.564+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3958014030 con 0x7f3964102790 2024-09-12T02:49:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:33 smithi196 ceph-mon[34587]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.775+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f396402d7c0 con 0x7f3964102790 2024-09-12T02:49:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.776+0000 7f39637fe640 1 -- 172.21.15.46:0/3643859698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7f395809c4b0 con 0x7f3964102790 2024-09-12T02:49:33.777 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:33.777 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:49:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 msgr2=0x7f394c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 0x7f394c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3954031440 tx=0x7f39540095f0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 msgr2=0x7f3964198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f395800cce0 tx=0x7f395800a620 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 shutdown_connections 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f394c0a2c10 0x7f394c0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3964106f10 0x7f396419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3964102790 0x7f3964198830 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 --2- 172.21.15.46:0/3643859698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3964101de0 0x7f39641982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 >> 172.21.15.46:0/3643859698 conn(0x7f39640fd8c0 msgr2=0x7f39640fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 shutdown_connections 2024-09-12T02:49:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:33.778+0000 7f396c54e640 1 -- 172.21.15.46:0/3643859698 wait complete. 2024-09-12T02:49:33.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:33 smithi145 ceph-mon[34674]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:49:34.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:49:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:34 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3643859698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:34 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3643859698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:34.859 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:34.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:34 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3643859698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 -- 172.21.15.46:0/4191963853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c1036c0 msgr2=0x7f8b9c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/4191963853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c10bbb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009900 tx=0x7f8b9002a480 comp rx=0 tx=0).stop 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 -- 172.21.15.46:0/4191963853 shutdown_connections 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/4191963853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c10c180 0x7f8b9c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/4191963853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c10bbb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/4191963853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c102da0 0x7f8b9c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 -- 172.21.15.46:0/4191963853 >> 172.21.15.46:0/4191963853 conn(0x7f8b9c0fc7a0 msgr2=0x7f8b9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:35.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 -- 172.21.15.46:0/4191963853 shutdown_connections 2024-09-12T02:49:35.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.373+0000 7f8ba1e63640 1 -- 172.21.15.46:0/4191963853 wait complete. 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.374+0000 7f8ba1e63640 1 Processor -- start 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.374+0000 7f8ba1e63640 1 -- start start 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c102da0 0x7f8b9c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111e50 con 0x7f8b9c10c180 2024-09-12T02:49:35.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111cd0 con 0x7f8b9c1036c0 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8ba1e63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111fd0 con 0x7f8b9c102da0 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44794/0 (socket says 172.21.15.46:44794) 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 -- 172.21.15.46:0/1473845412 learned_addr learned my addr 172.21.15.46:0/1473845412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9affd640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9b7fe640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c102da0 0x7f8b9c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 -- 172.21.15.46:0/1473845412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c102da0 msgr2=0x7f8b9c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c102da0 0x7f8b9c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 -- 172.21.15.46:0/1473845412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 msgr2=0x7f8b9c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 -- 172.21.15.46:0/1473845412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c19d310 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9affd640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.375+0000 7f8b9bfff640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b970 tx=0x7f8b9400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.376+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94019070 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.376+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c19d600 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.376+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b94014440 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.376+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b940075d0 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.376+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c1a4ee0 con 0x7f8b9c10c180 2024-09-12T02:49:35.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.377+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c103180 con 0x7f8b9c10c180 2024-09-12T02:49:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.378+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b9401c070 con 0x7f8b9c10c180 2024-09-12T02:49:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.379+0000 7f8b98ff9640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 0x7f8b840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.379+0000 7f8b9b7fe640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 0x7f8b840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.379+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8b940cea20 con 0x7f8b9c10c180 2024-09-12T02:49:35.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.379+0000 7f8b9b7fe640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 0x7f8b840a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005e10 tx=0x7f8b8c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:35.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.383+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b9409b5d0 con 0x7f8b9c10c180 2024-09-12T02:49:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.559+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b9c106c30 con 0x7f8b9c10c180 2024-09-12T02:49:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.560+0000 7f8b98ff9640 1 -- 172.21.15.46:0/1473845412 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24656 (secure 0 0 0) 0x7f8b9409aed0 con 0x7f8b9c10c180 2024-09-12T02:49:35.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:35.561 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:48:37.333173+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 msgr2=0x7f8b840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 0x7f8b840a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005e10 tx=0x7f8b8c005da0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 msgr2=0x7f8b9c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b970 tx=0x7f8b9400be30 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 shutdown_connections 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b9c10c180 0x7f8b9c19cc30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b9c1036c0 0x7f8b9c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f8b840a2c10 0x7f8b840a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 --2- 172.21.15.46:0/1473845412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b9c102da0 0x7f8b9c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.562+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 >> 172.21.15.46:0/1473845412 conn(0x7f8b9c0fc7a0 msgr2=0x7f8b9c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.563+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 shutdown_connections 2024-09-12T02:49:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:35.563+0000 7f8ba1e63640 1 -- 172.21.15.46:0/1473845412 wait complete. 2024-09-12T02:49:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:36 smithi046 ceph-mon[30951]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:36.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:36 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1473845412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:49:36.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:49:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:36 smithi196 ceph-mon[34587]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:36 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1473845412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:49:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:36 smithi145 ceph-mon[34674]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:36 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1473845412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:49:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 -- 172.21.15.46:0/506234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4794101df0 msgr2=0x7f47941021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 --2- 172.21.15.46:0/506234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4794101df0 0x7f47941021d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4784009900 tx=0x7f478402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 -- 172.21.15.46:0/506234798 shutdown_connections 2024-09-12T02:49:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 --2- 172.21.15.46:0/506234798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 0x7f479410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 --2- 172.21.15.46:0/506234798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47941027a0 0x7f47941067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 --2- 172.21.15.46:0/506234798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4794101df0 0x7f47941021d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 -- 172.21.15.46:0/506234798 >> 172.21.15.46:0/506234798 conn(0x7f47940fd8f0 msgr2=0x7f47940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 -- 172.21.15.46:0/506234798 shutdown_connections 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.189+0000 7f47999e0640 1 -- 172.21.15.46:0/506234798 wait complete. 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 Processor -- start 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 -- start start 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 0x7f47941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 0x7f479419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4794111ec0 con 0x7f47941027a0 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4794111d40 con 0x7f4794101df0 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47999e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112040 con 0x7f4794106f20 2024-09-12T02:49:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47927fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47927fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44812/0 (socket says 172.21.15.46:44812) 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.190+0000 7f47927fc640 1 -- 172.21.15.46:0/151726283 learned_addr learned my addr 172.21.15.46:0/151726283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f4792ffd640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 0x7f47941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47937fe640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 0x7f479419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 -- 172.21.15.46:0/151726283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 msgr2=0x7f479419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 0x7f479419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 -- 172.21.15.46:0/151726283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 msgr2=0x7f47941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 0x7f47941984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 -- 172.21.15.46:0/151726283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479419d470 con 0x7f47941027a0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f4792ffd640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 0x7f47941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47927fc640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f478800b4a0 tx=0x7f478800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788014070 con 0x7f47941027a0 2024-09-12T02:49:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479419d760 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4788010ac0 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788017c30 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.191+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a5040 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.192+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47941021d0 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.193+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4788017070 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.194+0000 7f47989de640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 0x7f477c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.194+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9121+0+0 (secure 0 0 0) 0x7f47880ceb50 con 0x7f47941027a0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.194+0000 7f4792ffd640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 0x7f477c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.194+0000 7f4792ffd640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 0x7f477c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4784005ab0 tx=0x7f4784000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.198+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f478809b730 con 0x7f47941027a0 2024-09-12T02:49:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.411+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f479410a100 con 0x7f47941027a0 2024-09-12T02:49:37.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:37 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:49:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.412+0000 7f47989de640 1 -- 172.21.15.46:0/151726283 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7f478809b030 con 0x7f47941027a0 2024-09-12T02:49:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:37.413 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:49:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 msgr2=0x7f477c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 0x7f477c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4784005ab0 tx=0x7f4784000bc0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 msgr2=0x7f4794198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f478800b4a0 tx=0x7f478800b960 comp rx=0 tx=0).stop 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 shutdown_connections 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f477c0a2b80 0x7f477c0a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4794106f20 0x7f479419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47941027a0 0x7f4794198a00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 --2- 172.21.15.46:0/151726283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4794101df0 0x7f47941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.414+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 >> 172.21.15.46:0/151726283 conn(0x7f47940fd8f0 msgr2=0x7f47940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.415+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 shutdown_connections 2024-09-12T02:49:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:37.415+0000 7f47999e0640 1 -- 172.21.15.46:0/151726283 wait complete. 2024-09-12T02:49:37.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:37 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:49:37.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:37 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:49:38.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:47:41.799879+0000', 'modified': '2024-09-12T02:48:21.888091+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24476, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1048929159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1048929159}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1048929159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/2556040092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 2556040092}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 2556040092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/3561512892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3561512892}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3561512892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6837/756659635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 756659635}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 756659635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.145:6839/1761070564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 1761070564}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 1761070564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6839/1262634008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1262634008}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1262634008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24478, 24476, 24460]} 2024-09-12T02:49:38.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T02:49:38.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:49:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:38 smithi196 ceph-mon[34587]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:38 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:49:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:38 smithi196 ceph-mon[34587]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:49:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:38 smithi196 ceph-mon[34587]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T02:49:38.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:38 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/151726283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:38.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:38 smithi046 ceph-mon[30951]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:38.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:38 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:49:38.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:38 smithi046 ceph-mon[30951]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:49:38.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:38 smithi046 ceph-mon[30951]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T02:49:38.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:38 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/151726283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:38 smithi145 ceph-mon[34674]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:38 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:49:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:38 smithi145 ceph-mon[34674]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:49:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:38 smithi145 ceph-mon[34674]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T02:49:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:38 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/151726283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 -- 172.21.15.46:0/3149380800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec102790 msgr2=0x7facec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 --2- 172.21.15.46:0/3149380800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec102790 0x7facec1067e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7face0009900 tx=0x7face002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 -- 172.21.15.46:0/3149380800 shutdown_connections 2024-09-12T02:49:38.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 --2- 172.21.15.46:0/3149380800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec106f10 0x7facec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 --2- 172.21.15.46:0/3149380800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec102790 0x7facec1067e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 --2- 172.21.15.46:0/3149380800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec101de0 0x7facec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 -- 172.21.15.46:0/3149380800 >> 172.21.15.46:0/3149380800 conn(0x7facec0fd8c0 msgr2=0x7facec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 -- 172.21.15.46:0/3149380800 shutdown_connections 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.873+0000 7facf1045640 1 -- 172.21.15.46:0/3149380800 wait complete. 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 Processor -- start 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 -- start start 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 0x7facec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 0x7facec19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facec111e20 con 0x7facec101de0 2024-09-12T02:49:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facec111ca0 con 0x7facec102790 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facf1045640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facec111fa0 con 0x7facec106f10 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.874+0000 7facead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44828/0 (socket says 172.21.15.46:44828) 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 -- 172.21.15.46:0/3934266367 learned_addr learned my addr 172.21.15.46:0/3934266367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facea59c640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 0x7facec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7faceb59e640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 0x7facec19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 -- 172.21.15.46:0/3934266367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 msgr2=0x7facec19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 0x7facec19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 -- 172.21.15.46:0/3934266367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 msgr2=0x7facec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 0x7facec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 -- 172.21.15.46:0/3934266367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec19d340 con 0x7facec101de0 2024-09-12T02:49:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facea59c640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 0x7facec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7faceb59e640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 0x7facec19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facead9d640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7facdc00d770 tx=0x7facdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facd3fff640 1 -- 172.21.15.46:0/3934266367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc017070 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facd3fff640 1 -- 172.21.15.46:0/3934266367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facdc0054d0 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec19d630 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facd3fff640 1 -- 172.21.15.46:0/3934266367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc004cf0 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.875+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec1a4f10 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facd3fff640 1 -- 172.21.15.46:0/3934266367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7facdc004e90 con 0x7facec101de0 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 Processor -- start 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 -- start start 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55897fba0090 con 0x55897fc3a1d0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55897fb47a70 con 0x55897fc398b0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facf341e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55897fb9c050 con 0x55897fc48fc0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.878+0000 7facead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facd3fff640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 0x7facd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44838/0 (socket says 172.21.15.46:44838) 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 -- 172.21.15.46:0/1398435746 learned_addr learned my addr 172.21.15.46:0/1398435746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7faceb59e640 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7faceb59e640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 0x7facd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facea59c640 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facd3fff640 1 -- 172.21.15.46:0/3934266367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7facdc0ceee0 con 0x7facec101de0 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 -- 172.21.15.46:0/1398435746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 msgr2=0x55897fc4b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc4b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 -- 172.21.15.46:0/1398435746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 msgr2=0x55897fc39c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc39c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 -- 172.21.15.46:0/1398435746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897fb54130 con 0x55897fc3a1d0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facead9d640 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7facdc09b0c0 tx=0x7facdc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b4b1e2fd455c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7faceb59e640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 0x7facd40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7face4005c40 tx=0x7face4009930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facd17fa640 1 -- 172.21.15.46:0/1398435746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc020070 con 0x55897fc3a1d0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facd17fa640 1 -- 172.21.15.46:0/1398435746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facdc009c30 con 0x55897fc3a1d0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.879+0000 7facd17fa640 1 -- 172.21.15.46:0/1398435746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc09f770 con 0x55897fc3a1d0 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- 172.21.15.46:0/1398435746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 msgr2=0x55897fc48a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7facdc09b0c0 tx=0x7facdc004740 comp rx=0 tx=0).stop 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- 172.21.15.46:0/1398435746 shutdown_connections 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc4b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc48a80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 --2- 172.21.15.46:0/1398435746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- 172.21.15.46:0/1398435746 >> 172.21.15.46:0/1398435746 conn(0x55897fc35f60 msgr2=0x55897fc36340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- 172.21.15.46:0/1398435746 shutdown_connections 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- 172.21.15.46:0/1398435746 wait complete. 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.880+0000 7facf341e740 1 -- start start 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 Processor -- start 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc3a1d0 0x55897fc45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55897fc4bb90 con 0x55897fc48fc0 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55897fb5af20 con 0x55897fc398b0 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55897fc4bd10 con 0x55897fc3a1d0 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facf341e740 10 client.0 _ll_register_callbacks cb 0x55897fb24030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44842/0 (socket says 172.21.15.46:44842) 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 -- 172.21.15.46:0/2369399065 learned_addr learned my addr 172.21.15.46:0/2369399065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facea59c640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc3a1d0 0x55897fc45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facead9d640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 -- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc3a1d0 msgr2=0x55897fc45c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc3a1d0 0x55897fc45c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 -- 172.21.15.46:0/2369399065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 msgr2=0x55897fc488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7faceb59e640 1 -- 172.21.15.46:0/2369399065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897f9112b0 con 0x55897fc48fc0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.881+0000 7facead9d640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7faceb59e640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7face4014270 tx=0x7face4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eca44e9e7bc0ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4026070 con 0x55897fc48fc0 2024-09-12T02:49:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7face4004d20 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55897fbad600 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7face4031050 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 10 client.15051 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55897fc57540 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55897fbad600 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7face4005620 con 0x55897fc48fc0 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 10 client.15051 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 10 client.15051 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 20 client.15051 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fc59b80 0x55897fc5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 4 client.15051 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:49:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.882+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 --> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897fba7b10 con 0x55897fc59b80 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.883+0000 7facead9d640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fc59b80 0x55897fc5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.883+0000 7facead9d640 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fc59b80 0x55897fc5aff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1431f43e666cebef server_cookie=751ea9aa8bb10b2a in_seq=0 out_seq=0 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.883+0000 7facceff5640 10 client.15051 ms_handle_connect on v2:172.21.15.196:6836/756659635 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facceff5640 1 -- 172.21.15.46:0/2369399065 <== mds.2 v2:172.21.15.196:6836/756659635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55897fba7b10 con 0x55897fc59b80 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facceff5640 10 client.15051 handle_command_reply: tid=0 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 client.15051 shutdown 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 msgr2=0x55897fc46150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7face4014270 tx=0x7face4009650 comp rx=0 tx=0).stop 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 shutdown_connections 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fc59b80 0x55897fc5aff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc48fc0 0x55897fc46150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc3a1d0 0x55897fc45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 --2- 172.21.15.46:0/2369399065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facceff5640 0 client.15051 ms_handle_reset on v2:172.21.15.196:6836/756659635 2024-09-12T02:49:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 >> 172.21.15.46:0/2369399065 conn(0x55897fc35f60 msgr2=0x55897fc3bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 shutdown_connections 2024-09-12T02:49:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 1 -- 172.21.15.46:0/2369399065 wait complete. 2024-09-12T02:49:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.884+0000 7facf341e740 20 client.15051 trim_cache size 0 max 16384 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 Processor -- start 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 -- start start 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc63810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55897fc59330 con 0x55897fc3a1d0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55897fc59a00 con 0x55897fc398b0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facf341e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55897fc5a0d0 con 0x55897fc48fc0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.902+0000 7facea59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44848/0 (socket says 172.21.15.46:44848) 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 -- 172.21.15.46:0/1253820455 learned_addr learned my addr 172.21.15.46:0/1253820455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facead9d640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7faceb59e640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc63810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 -- 172.21.15.46:0/1253820455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 msgr2=0x55897fc63810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc63810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 -- 172.21.15.46:0/1253820455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 msgr2=0x55897fc5aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc5aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 -- 172.21.15.46:0/1253820455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897fc484e0 con 0x55897fc3a1d0 2024-09-12T02:49:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7faceb59e640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc63810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facea59c640 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7face0033000 tx=0x7face0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6091fb4b1dcbaffb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facd0ff9640 1 -- 172.21.15.46:0/1253820455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0035040 con 0x55897fc3a1d0 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facd0ff9640 1 -- 172.21.15.46:0/1253820455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face00317e0 con 0x55897fc3a1d0 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.903+0000 7facd0ff9640 1 -- 172.21.15.46:0/1253820455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00053b0 con 0x55897fc3a1d0 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- 172.21.15.46:0/1253820455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 msgr2=0x55897fc632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7face0033000 tx=0x7face0032e10 comp rx=0 tx=0).stop 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- 172.21.15.46:0/1253820455 shutdown_connections 2024-09-12T02:49:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc63810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc632d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 --2- 172.21.15.46:0/1253820455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- 172.21.15.46:0/1253820455 >> 172.21.15.46:0/1253820455 conn(0x55897fc35f60 msgr2=0x55897fc444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- 172.21.15.46:0/1253820455 shutdown_connections 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- 172.21.15.46:0/1253820455 wait complete. 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 -- start start 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.904+0000 7facf341e740 1 Processor -- start 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc63860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc42ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55897fc5ae70 con 0x55897fc3a1d0 2024-09-12T02:49:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55897fc5a7a0 con 0x55897fc398b0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55897fc5b540 con 0x55897fc48fc0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facf341e740 10 client.0 _ll_register_callbacks cb 0x7face4009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44860/0 (socket says 172.21.15.46:44860) 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 -- 172.21.15.46:0/2307579126 learned_addr learned my addr 172.21.15.46:0/2307579126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facead9d640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7faceb59e640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc42ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 -- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 msgr2=0x55897fc42ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc42ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 -- 172.21.15.46:0/2307579126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 msgr2=0x55897fc63860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc63860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 -- 172.21.15.46:0/2307579126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897fc433d0 con 0x55897fc3a1d0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facea59c640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7face0000c00 tx=0x7face0005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98f71ecaac6531a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.905+0000 7facceff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0035030 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7face00046c0 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face00317e0 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7face0057050 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 10 client.15057 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55897fcc4290 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face00317e0 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7face0044070 con 0x55897fc3a1d0 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 10 client.15057 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 20 client.15057 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fcc8900 0x55897fcc7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 4 client.15057 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:49:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.906+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 --> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] -- command(tid 0: {"prefix": "status"}) -- 0x55897fb240e0 con 0x55897fcc8900 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facead9d640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fcc8900 0x55897fcc7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facead9d640 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fcc8900 0x55897fcc7610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30290def6e004a6e server_cookie=e4281ede7b6883fc in_seq=0 out_seq=0 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facceff5640 10 client.15057 ms_handle_connect on v2:172.21.15.196:6836/756659635 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facceff5640 1 -- 172.21.15.46:0/2307579126 <== mds.2 v2:172.21.15.196:6836/756659635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55897fb240e0 con 0x55897fcc8900 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facceff5640 10 client.15057 handle_command_reply: tid=0 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 client.15057 shutdown 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 msgr2=0x55897fc427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7face0000c00 tx=0x7face0005360 comp rx=0 tx=0).stop 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 shutdown_connections 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x55897fcc8900 0x55897fcc7610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55897fc48fc0 0x55897fc42ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55897fc3a1d0 0x55897fc427a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 --2- 172.21.15.46:0/2307579126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55897fc398b0 0x55897fc63860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 >> 172.21.15.46:0/2307579126 conn(0x55897fc35f60 msgr2=0x55897fc41240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.907+0000 7facceff5640 0 client.15057 ms_handle_reset on v2:172.21.15.196:6836/756659635 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 shutdown_connections 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf341e740 1 -- 172.21.15.46:0/2307579126 wait complete. 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf341e740 20 client.15057 trim_cache size 0 max 16384 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24460, 2024-09-12T02:49:38.913 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 79.682021355000003, 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 142.45950723000001 2024-09-12T02:49:38.914 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 msgr2=0x7facd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 0x7facd40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7face4005c40 tx=0x7face4009930 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 msgr2=0x7facec198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7facdc00d770 tx=0x7facdc00dc30 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 shutdown_connections 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7facd40a2b80 0x7facd40a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facec106f10 0x7facec19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facec102790 0x7facec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 --2- 172.21.15.46:0/3934266367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facec101de0 0x7facec198390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 >> 172.21.15.46:0/3934266367 conn(0x7facec0fd8c0 msgr2=0x7facec0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 shutdown_connections 2024-09-12T02:49:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:38.908+0000 7facf1045640 1 -- 172.21.15.46:0/3934266367 wait complete. 2024-09-12T02:49:39.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:39 smithi196 ceph-mon[34587]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T02:49:39.525 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:39 smithi046 ceph-mon[30951]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T02:49:39.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:39 smithi145 ceph-mon[34674]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T02:49:39.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:49:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:40.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:40 smithi046 ceph-mon[30951]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-12T02:49:40.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:40 smithi196 ceph-mon[34587]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-12T02:49:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 -- 172.21.15.46:0/1989007092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 msgr2=0x7fba5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 --2- 172.21.15.46:0/1989007092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c1067e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba5002a410 comp rx=0 tx=0).stop 2024-09-12T02:49:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 -- 172.21.15.46:0/1989007092 shutdown_connections 2024-09-12T02:49:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 --2- 172.21.15.46:0/1989007092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 0x7fba5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 --2- 172.21.15.46:0/1989007092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c1067e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 --2- 172.21.15.46:0/1989007092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 -- 172.21.15.46:0/1989007092 >> 172.21.15.46:0/1989007092 conn(0x7fba5c0fd8c0 msgr2=0x7fba5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 -- 172.21.15.46:0/1989007092 shutdown_connections 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.471+0000 7fba62182640 1 -- 172.21.15.46:0/1989007092 wait complete. 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.472+0000 7fba62182640 1 Processor -- start 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.472+0000 7fba62182640 1 -- start start 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 0x7fba5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c111e20 con 0x7fba5c102790 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c111ca0 con 0x7fba5c101de0 2024-09-12T02:49:40.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba62182640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c111fa0 con 0x7fba5c106f10 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44880/0 (socket says 172.21.15.46:44880) 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 -- 172.21.15.46:0/2191274246 learned_addr learned my addr 172.21.15.46:0/2191274246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 0x7fba5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 -- 172.21.15.46:0/2191274246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 msgr2=0x7fba5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 0x7fba5c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 -- 172.21.15.46:0/2191274246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 msgr2=0x7fba5c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 -- 172.21.15.46:0/2191274246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1a6820 con 0x7fba5c102790 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:40.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.473+0000 7fba5affd640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba50005940 tx=0x7fba500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.474+0000 7fba58ff9640 1 -- 172.21.15.46:0/2191274246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5003f030 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.474+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1a6ab0 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.474+0000 7fba58ff9640 1 -- 172.21.15.46:0/2191274246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba500044c0 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.474+0000 7fba58ff9640 1 -- 172.21.15.46:0/2191274246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50035070 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.474+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1a6fe0 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.476+0000 7fba58ff9640 1 -- 172.21.15.46:0/2191274246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba50042030 con 0x7fba5c102790 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.476+0000 7fba6455b740 1 Processor -- start 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.476+0000 7fba6455b740 1 -- start start 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 0x55555c59bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59c2f0 0x55555c5a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55555c502090 con 0x55555c5a6bb0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55555c4a9a70 con 0x55555c59c2f0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba6455b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55555c4fe050 con 0x55555c59b9d0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44886/0 (socket says 172.21.15.46:44886) 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5affd640 1 -- 172.21.15.46:0/3973070860 learned_addr learned my addr 172.21.15.46:0/3973070860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 0x55555c59bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59c2f0 0x55555c5a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba58ff9640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 -- 172.21.15.46:0/3973070860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59c2f0 msgr2=0x55555c5a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59c2f0 0x55555c5a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 -- 172.21.15.46:0/3973070860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 msgr2=0x55555c5a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 -- 172.21.15.46:0/3973070860 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55555c4b6130 con 0x55555c59b9d0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5affd640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.477+0000 7fba58ff9640 1 -- 172.21.15.46:0/2191274246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fba500ef790 con 0x7fba5c102790 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.478+0000 7fba5bfff640 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 0x55555c59bdb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba54009fa0 tx=0x7fba540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed04632fc20c500 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.478+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 0x7fba440a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba54022000 tx=0x7fba54023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.478+0000 7fba41ffb640 1 -- 172.21.15.46:0/3973070860 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54017050 con 0x55555c59b9d0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.478+0000 7fba41ffb640 1 -- 172.21.15.46:0/3973070860 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba54003b90 con 0x55555c59b9d0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.478+0000 7fba41ffb640 1 -- 172.21.15.46:0/3973070860 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5401b8f0 con 0x55555c59b9d0 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- 172.21.15.46:0/3973070860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 msgr2=0x55555c59bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 0x55555c59bdb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba54009fa0 tx=0x7fba540118d0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- 172.21.15.46:0/3973070860 shutdown_connections 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c5a6bb0 0x55555c5a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59c2f0 0x55555c5a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 --2- 172.21.15.46:0/3973070860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59b9d0 0x55555c59bdb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- 172.21.15.46:0/3973070860 >> 172.21.15.46:0/3973070860 conn(0x55555c597f60 msgr2=0x55555c598340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- 172.21.15.46:0/3973070860 shutdown_connections 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- 172.21.15.46:0/3973070860 wait complete. 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 -- start start 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.479+0000 7fba6455b740 1 Processor -- start 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55555c5a9780 con 0x55555c59c2f0 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55555c4bcf20 con 0x55555c59b9d0 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55555c5a9900 con 0x55555c5a6bb0 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba6455b740 10 client.0 _ll_register_callbacks cb 0x55555c486030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44900/0 (socket says 172.21.15.46:44900) 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 -- 172.21.15.46:0/2162906423 learned_addr learned my addr 172.21.15.46:0/2162906423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 -- 172.21.15.46:0/2162906423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 msgr2=0x55555c5a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 -- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 msgr2=0x55555c5a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 -- 172.21.15.46:0/2162906423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55555c2732b0 con 0x55555c59c2f0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba5affd640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba50031440 tx=0x7fba50005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b092a0a82e3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba3affd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.480+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50049070 con 0x55555c59c2f0 2024-09-12T02:49:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fba500b82e0 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba54003b90 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba500bb7b0 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 10 client.15069 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55555c5b50d0 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba54003b90 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fba500bbed0 con 0x55555c59c2f0 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 10 client.15069 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 20 client.15069 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c5b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 4 client.15069 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:49:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.481+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 --> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55555c50f600 con 0x55555c5b7720 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.482+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c5b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.482+0000 7fba3affd640 10 client.15069 ms_handle_connect on v2:172.21.15.145:6838/1761070564 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.482+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c5b8b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec3ef266004007f3 server_cookie=833d8b2c6e0410e1 in_seq=0 out_seq=0 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba3affd640 1 -- 172.21.15.46:0/2162906423 <== mds.1 v2:172.21.15.145:6838/1761070564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55555c50f600 con 0x55555c5b7720 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba3affd640 10 client.15069 handle_command_reply: tid=0 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 client.15069 shutdown 2024-09-12T02:49:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 msgr2=0x55555c5a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba50031440 tx=0x7fba50005a20 comp rx=0 tx=0).stop 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 shutdown_connections 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c5b8b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba3affd640 0 client.15069 ms_handle_reset on v2:172.21.15.145:6838/1761070564 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c5a37d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 --2- 172.21.15.46:0/2162906423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 >> 172.21.15.46:0/2162906423 conn(0x55555c597f60 msgr2=0x55555c59dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 shutdown_connections 2024-09-12T02:49:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 1 -- 172.21.15.46:0/2162906423 wait complete. 2024-09-12T02:49:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.483+0000 7fba6455b740 20 client.15069 trim_cache size 0 max 16384 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 Processor -- start 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 -- start start 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b6ed0 con 0x55555c59c2f0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b75a0 con 0x55555c59b9d0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba6455b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b7c70 con 0x55555c5a6bb0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44916/0 (socket says 172.21.15.46:44916) 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 -- 172.21.15.46:0/2303323552 learned_addr learned my addr 172.21.15.46:0/2303323552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 -- 172.21.15.46:0/2303323552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 msgr2=0x55555c5b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 -- 172.21.15.46:0/2303323552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 msgr2=0x55555c5b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5affd640 1 -- 172.21.15.46:0/2303323552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55555c5a60a0 con 0x55555c59c2f0 2024-09-12T02:49:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.502+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:40.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.503+0000 7fba5affd640 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba500b80d0 tx=0x7fba500ba180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8238540a79a0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.503+0000 7fba417fa640 1 -- 172.21.15.46:0/2303323552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50049030 con 0x55555c59c2f0 2024-09-12T02:49:40.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.503+0000 7fba417fa640 1 -- 172.21.15.46:0/2303323552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba500bc370 con 0x55555c59c2f0 2024-09-12T02:49:40.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.503+0000 7fba417fa640 1 -- 172.21.15.46:0/2303323552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500bb5d0 con 0x55555c59c2f0 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- 172.21.15.46:0/2303323552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 msgr2=0x55555c592ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba500b80d0 tx=0x7fba500ba180 comp rx=0 tx=0).stop 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- 172.21.15.46:0/2303323552 shutdown_connections 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c5a6bb0 0x55555c5b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59c2f0 0x55555c592ce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 --2- 172.21.15.46:0/2303323552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c59b9d0 0x55555c5b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- 172.21.15.46:0/2303323552 >> 172.21.15.46:0/2303323552 conn(0x55555c597f60 msgr2=0x55555c598340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- 172.21.15.46:0/2303323552 shutdown_connections 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- 172.21.15.46:0/2303323552 wait complete. 2024-09-12T02:49:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 -- start start 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.504+0000 7fba6455b740 1 Processor -- start 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59c2f0 0x55555c5bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 0x55555c61fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b8a10 con 0x55555c59b9d0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b8340 con 0x55555c5a6bb0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55555c5b90e0 con 0x55555c59c2f0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba6455b740 10 client.0 _ll_register_callbacks cb 0x7fba5003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44932/0 (socket says 172.21.15.46:44932) 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 -- 172.21.15.46:0/2779463475 learned_addr learned my addr 172.21.15.46:0/2779463475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5affd640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59c2f0 0x55555c5bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 0x55555c61fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 -- 172.21.15.46:0/2779463475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59c2f0 msgr2=0x55555c5bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59c2f0 0x55555c5bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 -- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 msgr2=0x55555c61fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 0x55555c61fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.505+0000 7fba5b7fe640 1 -- 172.21.15.46:0/2779463475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55555c620360 con 0x55555c59b9d0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba5bfff640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 0x55555c61fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba5b7fe640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009c90 tx=0x7fba4c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a6910f7c649dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba3affd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c01e790 con 0x55555c59b9d0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fba4c01e930 con 0x55555c59b9d0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba500bc370 con 0x55555c59b9d0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fba4c01eb30 con 0x55555c59b9d0 2024-09-12T02:49:40.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba6455b740 10 client.15075 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.506+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55555c621580 con 0x55555c59b9d0 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba500bc370 con 0x55555c59b9d0 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fba4c026400 con 0x55555c59b9d0 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 10 client.15075 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 20 client.15075 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c624f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 4 client.15075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 --> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] -- command(tid 0: {"prefix": "status"}) -- 0x55555c509b10 con 0x55555c5b7720 2024-09-12T02:49:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.507+0000 7fba5affd640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c624f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba3affd640 10 client.15075 ms_handle_connect on v2:172.21.15.145:6838/1761070564 2024-09-12T02:49:40.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba5affd640 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c624f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34bf97a749fd4653 server_cookie=8acde0d97994a54b in_seq=0 out_seq=0 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba3affd640 1 -- 172.21.15.46:0/2779463475 <== mds.1 v2:172.21.15.145:6838/1761070564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55555c509b10 con 0x55555c5b7720 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba3affd640 10 client.15075 handle_command_reply: tid=0 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 client.15075 shutdown 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 msgr2=0x55555c5bc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009c90 tx=0x7fba4c0039c0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 shutdown_connections 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x55555c5b7720 0x55555c624f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55555c5a6bb0 0x55555c61fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55555c59c2f0 0x55555c5bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 --2- 172.21.15.46:0/2779463475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55555c59b9d0 0x55555c5bc160 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 >> 172.21.15.46:0/2779463475 conn(0x55555c597f60 msgr2=0x55555c592ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba3affd640 0 client.15075 ms_handle_reset on v2:172.21.15.145:6838/1761070564 2024-09-12T02:49:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.508+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 shutdown_connections 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba6455b740 1 -- 172.21.15.46:0/2779463475 wait complete. 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba6455b740 20 client.15075 trim_cache size 0 max 16384 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24476, 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:49:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 82.294943918000001, 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 129.47107568600001 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 msgr2=0x7fba440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 0x7fba440a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba54022000 tx=0x7fba54023000 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 msgr2=0x7fba5c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba50005940 tx=0x7fba500040c0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 shutdown_connections 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5c106f10 0x7fba5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba5c102790 0x7fba5c074a30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 --2- 172.21.15.46:0/2191274246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba5c101de0 0x7fba5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 >> 172.21.15.46:0/2191274246 conn(0x7fba5c0fd8c0 msgr2=0x7fba5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 shutdown_connections 2024-09-12T02:49:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:40.509+0000 7fba62182640 1 -- 172.21.15.46:0/2191274246 wait complete. 2024-09-12T02:49:40.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:40 smithi145 ceph-mon[34674]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-12T02:49:41.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:49:41.491 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 -- 172.21.15.46:0/3507875536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101de0 msgr2=0x7f79841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 --2- 172.21.15.46:0/3507875536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101de0 0x7f79841021c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7978009900 tx=0x7f797802a410 comp rx=0 tx=0).stop 2024-09-12T02:49:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 -- 172.21.15.46:0/3507875536 shutdown_connections 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 --2- 172.21.15.46:0/3507875536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 --2- 172.21.15.46:0/3507875536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984102790 0x7f79841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 --2- 172.21.15.46:0/3507875536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101de0 0x7f79841021c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 -- 172.21.15.46:0/3507875536 >> 172.21.15.46:0/3507875536 conn(0x7f79840fd8c0 msgr2=0x7f79840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 -- 172.21.15.46:0/3507875536 shutdown_connections 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.024+0000 7f798b0e3640 1 -- 172.21.15.46:0/3507875536 wait complete. 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 Processor -- start 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 -- start start 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984101de0 0x7f7984198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7984111de0 con 0x7f7984102790 2024-09-12T02:49:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7984111c60 con 0x7f7984106f10 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f798b0e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7984111f60 con 0x7f7984101de0 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44956/0 (socket says 172.21.15.46:44956) 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 -- 172.21.15.46:0/44840262 learned_addr learned my addr 172.21.15.46:0/44840262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7988e7f640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984101de0 0x7f7984198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7989680640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 -- 172.21.15.46:0/44840262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984101de0 msgr2=0x7f7984198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984101de0 0x7f7984198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 -- 172.21.15.46:0/44840262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 msgr2=0x7f798419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7983fff640 1 -- 172.21.15.46:0/44840262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798419d2c0 con 0x7f7984102790 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.025+0000 7f7989680640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f7983fff640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f797000c920 tx=0x7f797000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f7981ffb640 1 -- 172.21.15.46:0/44840262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970007c70 con 0x7f7984102790 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f7981ffb640 1 -- 172.21.15.46:0/44840262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7970007e10 con 0x7f7984102790 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f7981ffb640 1 -- 172.21.15.46:0/44840262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970018820 con 0x7f7984102790 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798419d5b0 con 0x7f7984102790 2024-09-12T02:49:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.026+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a4e90 con 0x7f7984102790 2024-09-12T02:49:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f7981ffb640 1 -- 172.21.15.46:0/44840262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f79700189c0 con 0x7f7984102790 2024-09-12T02:49:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 Processor -- start 2024-09-12T02:49:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 -- start start 2024-09-12T02:49:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a109229090 con 0x55a1092cdbb0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a1091d0a70 con 0x55a1092c29d0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.029+0000 7f798d4bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a109225050 con 0x55a1092c32f0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7988e7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7988e7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:33514/0 (socket says 172.21.15.46:33514) 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7988e7f640 1 -- 172.21.15.46:0/2747952277 learned_addr learned my addr 172.21.15.46:0/2747952277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7981ffb640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 0x7f79740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 -- 172.21.15.46:0/2747952277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 msgr2=0x55a1092cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 -- 172.21.15.46:0/2747952277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 msgr2=0x55a1092cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 -- 172.21.15.46:0/2747952277 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1091dd130 con 0x55a1092c29d0 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7983fff640 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 0x7f79740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7981ffb640 1 -- 172.21.15.46:0/44840262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f79700cf4f0 con 0x7f7984102790 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.030+0000 7f7989680640 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092c2db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f797c0104a0 tx=0x7f797c010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=accb3aae71c90060 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.031+0000 7f7989680640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 0x7f79740a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f797c021000 tx=0x7f797c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.031+0000 7f796effd640 1 -- 172.21.15.46:0/2747952277 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c019070 con 0x55a1092c29d0 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.031+0000 7f796effd640 1 -- 172.21.15.46:0/2747952277 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797c022070 con 0x55a1092c29d0 2024-09-12T02:49:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.031+0000 7f796effd640 1 -- 172.21.15.46:0/2747952277 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c0054b0 con 0x55a1092c29d0 2024-09-12T02:49:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- 172.21.15.46:0/2747952277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 msgr2=0x55a1092c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092c2db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f797c0104a0 tx=0x7f797c010960 comp rx=0 tx=0).stop 2024-09-12T02:49:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- 172.21.15.46:0/2747952277 shutdown_connections 2024-09-12T02:49:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2747952277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092c2db0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- 172.21.15.46:0/2747952277 >> 172.21.15.46:0/2747952277 conn(0x55a1092bef60 msgr2=0x55a1092bf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- 172.21.15.46:0/2747952277 shutdown_connections 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- 172.21.15.46:0/2747952277 wait complete. 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 -- start start 2024-09-12T02:49:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.032+0000 7f798d4bc740 1 Processor -- start 2024-09-12T02:49:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a1092d4b90 con 0x55a1092c32f0 2024-09-12T02:49:42.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a1091e3f20 con 0x55a1092cdbb0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1092d4d10 con 0x55a1092c29d0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f798d4bc740 10 client.0 _ll_register_callbacks cb 0x55a1091ad030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7988e7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44986/0 (socket says 172.21.15.46:44986) 2024-09-12T02:49:42.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7983fff640 1 -- 172.21.15.46:0/1782621077 learned_addr learned my addr 172.21.15.46:0/1782621077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 -- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 msgr2=0x55a1092cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 -- 172.21.15.46:0/1782621077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 msgr2=0x55a1092ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7989680640 1 -- 172.21.15.46:0/1782621077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a108f9a2b0 con 0x55a1092cdbb0 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7988e7f640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:42.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.033+0000 7f7983fff640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:49:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f7989680640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092cad60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f797c0104a0 tx=0x7f797c005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac692295a31d99d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f7963fff640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:49:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c019070 con 0x55a1092cdbb0 2024-09-12T02:49:42.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f797c00e900 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797c022070 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f797c00eb00 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f798d4bc740 10 client.24815 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.034+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a1092e0510 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797c022070 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f797c02d070 con 0x55a1092cdbb0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 10 client.24815 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 10 client.24815 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 20 client.24815 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a1092e28c0 0x55a1092e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 4 client.24815 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a109236600 con 0x55a1092e28c0 2024-09-12T02:49:42.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.035+0000 7f7988e7f640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a1092e28c0 0x55a1092e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.036+0000 7f7988e7f640 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a1092e28c0 0x55a1092e3d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b941eac04a0de1 server_cookie=9caea2052766125a in_seq=0 out_seq=0 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.036+0000 7f7963fff640 10 client.24815 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.036+0000 7f7963fff640 1 -- 172.21.15.46:0/1782621077 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a109236600 con 0x55a1092e28c0 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.036+0000 7f7963fff640 10 client.24815 handle_command_reply: tid=0 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.036+0000 7f798d4bc740 1 client.24815 shutdown 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 msgr2=0x55a1092cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092cad60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f797c0104a0 tx=0x7f797c005c70 comp rx=0 tx=0).stop 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 shutdown_connections 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a1092e28c0 0x55a1092e3d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092cad60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 --2- 172.21.15.46:0/1782621077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 >> 172.21.15.46:0/1782621077 conn(0x55a1092bef60 msgr2=0x55a1092c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f7963fff640 0 client.24815 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.037+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 shutdown_connections 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.038+0000 7f798d4bc740 1 -- 172.21.15.46:0/1782621077 wait complete. 2024-09-12T02:49:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.038+0000 7f798d4bc740 20 client.24815 trim_cache size 0 max 16384 2024-09-12T02:49:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.055+0000 7f798d4bc740 1 Processor -- start 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 -- start start 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e2070 con 0x55a1092c32f0 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e2740 con 0x55a1092cdbb0 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f798d4bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e2e10 con 0x55a1092c29d0 2024-09-12T02:49:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44992/0 (socket says 172.21.15.46:44992) 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 -- 172.21.15.46:0/3660622546 learned_addr learned my addr 172.21.15.46:0/3660622546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7988e7f640 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7989680640 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 -- 172.21.15.46:0/3660622546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 msgr2=0x55a1092e28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092e28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 -- 172.21.15.46:0/3660622546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 msgr2=0x55a1092e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.056+0000 7f7983fff640 1 -- 172.21.15.46:0/3660622546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1092c3b70 con 0x55a1092c32f0 2024-09-12T02:49:42.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.057+0000 7f7983fff640 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f79700cccc0 tx=0x7f79700d5060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27220152b68244c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.057+0000 7f796e7fc640 1 -- 172.21.15.46:0/3660622546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000fb00 con 0x55a1092c32f0 2024-09-12T02:49:42.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.057+0000 7f796e7fc640 1 -- 172.21.15.46:0/3660622546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797009c0d0 con 0x55a1092c32f0 2024-09-12T02:49:42.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.057+0000 7f796e7fc640 1 -- 172.21.15.46:0/3660622546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700cf270 con 0x55a1092c32f0 2024-09-12T02:49:42.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- 172.21.15.46:0/3660622546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 msgr2=0x55a1092e3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f79700cccc0 tx=0x7f79700d5060 comp rx=0 tx=0).stop 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- 172.21.15.46:0/3660622546 shutdown_connections 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092cdbb0 0x55a1092e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092c32f0 0x55a1092e3660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 --2- 172.21.15.46:0/3660622546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c29d0 0x55a1092e28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- 172.21.15.46:0/3660622546 >> 172.21.15.46:0/3660622546 conn(0x55a1092bef60 msgr2=0x55a1092e1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- 172.21.15.46:0/3660622546 shutdown_connections 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- 172.21.15.46:0/3660622546 wait complete. 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 -- start start 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.058+0000 7f798d4bc740 1 Processor -- start 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e3bb0 con 0x55a1092cdbb0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e34e0 con 0x55a1092c29d0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1092e4280 con 0x55a1092c32f0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f798d4bc740 10 client.0 _ll_register_callbacks cb 0x7f79700d2ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44996/0 (socket says 172.21.15.46:44996) 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 -- 172.21.15.46:0/2657444389 learned_addr learned my addr 172.21.15.46:0/2657444389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7988e7f640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7983fff640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 -- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 msgr2=0x55a1092e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 -- 172.21.15.46:0/2657444389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 msgr2=0x55a1092ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.059+0000 7f7989680640 1 -- 172.21.15.46:0/2657444389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1092e5b20 con 0x55a1092cdbb0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7989680640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f797c02f550 tx=0x7f797c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09076e9769fa4e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c019040 con 0x55a1092cdbb0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f797c02fb30 con 0x55a1092cdbb0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797009c0d0 con 0x55a1092cdbb0 2024-09-12T02:49:42.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f797c045050 con 0x55a1092cdbb0 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 10 client.15093 fetch_fsmap learned FSMap version 24 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a10934be70 con 0x55a1092cdbb0 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f797009c0d0 con 0x55a1092cdbb0 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f797c02d040 con 0x55a1092cdbb0 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 10 client.15093 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.060+0000 7f798d4bc740 20 client.15093 populate_metadata read hostname 'smithi046' 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a109350980 0x55a10934f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f798d4bc740 4 client.15093 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "status"}) -- 0x55a109230b10 con 0x55a109350980 2024-09-12T02:49:42.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f7988e7f640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a109350980 0x55a10934f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f7988e7f640 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a109350980 0x55a10934f690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a3b05ca2935c78 server_cookie=cfb83461ce10afe3 in_seq=0 out_seq=0 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.061+0000 7f7963fff640 10 client.15093 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f7963fff640 1 -- 172.21.15.46:0/2657444389 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a109230b10 con 0x55a109350980 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f7963fff640 10 client.15093 handle_command_reply: tid=0 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 client.15093 shutdown 2024-09-12T02:49:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 msgr2=0x55a1092e54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f797c02f550 tx=0x7f797c00ea10 comp rx=0 tx=0).stop 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 shutdown_connections 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a109350980 0x55a10934f690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f7963fff640 0 client.15093 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1092cdbb0 0x55a1092e54b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1092c32f0 0x55a1092e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 --2- 172.21.15.46:0/2657444389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a1092c29d0 0x55a1092ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 >> 172.21.15.46:0/2657444389 conn(0x55a1092bef60 msgr2=0x55a1092bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 shutdown_connections 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 1 -- 172.21.15.46:0/2657444389 wait complete. 2024-09-12T02:49:42.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.062+0000 7f798d4bc740 20 client.15093 trim_cache size 0 max 16384 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24478, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 119.96029076799999, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:49:42.069 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 140.207795234 2024-09-12T02:49:42.070 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:49:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 msgr2=0x7f79740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 0x7f79740a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f797c021000 tx=0x7f797c01f000 comp rx=0 tx=0).stop 2024-09-12T02:49:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 msgr2=0x7f7984198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f797000c920 tx=0x7f797000cde0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 shutdown_connections 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7984106f10 0x7f798419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984102790 0x7f7984198850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f79740a2b80 0x7f79740a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 --2- 172.21.15.46:0/44840262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7984101de0 0x7f7984198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 >> 172.21.15.46:0/44840262 conn(0x7f79840fd8c0 msgr2=0x7f79840fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 shutdown_connections 2024-09-12T02:49:42.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:42.063+0000 7f798b0e3640 1 -- 172.21.15.46:0/44840262 wait complete. 2024-09-12T02:49:42.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:42 smithi046 ceph-mon[30951]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:42.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:42 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:42.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:42 smithi196 ceph-mon[34587]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:42.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:42 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:42.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:42 smithi145 ceph-mon[34674]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:42.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:42 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:42.754 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T02:49:42.754 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:49:43.005 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:49:43.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T02:49:43.653 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.142+0000 7fbabbf42640 1 -- 172.21.15.46:0/1745294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 msgr2=0x7fbab4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.142+0000 7fbabbf42640 1 --2- 172.21.15.46:0/1745294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab4102150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009900 tx=0x7fbaa402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 -- 172.21.15.46:0/1745294022 shutdown_connections 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 --2- 172.21.15.46:0/1745294022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4102690 0x7fbab4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 --2- 172.21.15.46:0/1745294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab4102150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 --2- 172.21.15.46:0/1745294022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4100ad0 0x7fbab4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 -- 172.21.15.46:0/1745294022 >> 172.21.15.46:0/1745294022 conn(0x7fbab40fc280 msgr2=0x7fbab40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 -- 172.21.15.46:0/1745294022 shutdown_connections 2024-09-12T02:49:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 -- 172.21.15.46:0/1745294022 wait complete. 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 Processor -- start 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.143+0000 7fbabbf42640 1 -- start start 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4100ad0 0x7fbab4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4102690 0x7fbab419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbab410d730 con 0x7fbab4101cd0 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbab410d5b0 con 0x7fbab4102690 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbabbf42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbab410d8b0 con 0x7fbab4100ad0 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45018/0 (socket says 172.21.15.46:45018) 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 -- 172.21.15.46:0/2092963288 learned_addr learned my addr 172.21.15.46:0/2092963288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab9cde640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4100ad0 0x7fbab4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbaba4df640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4102690 0x7fbab419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 -- 172.21.15.46:0/2092963288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4100ad0 msgr2=0x7fbab4197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4100ad0 0x7fbab4197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 -- 172.21.15.46:0/2092963288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4102690 msgr2=0x7fbab419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4102690 0x7fbab419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 -- 172.21.15.46:0/2092963288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab419f9c0 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.144+0000 7fbab94dd640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004770 tx=0x7fbaa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.147+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa403f030 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.147+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa4003cb0 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.147+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab419fc50 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.147+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4035070 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.147+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab41a0180 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.148+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab4107f10 con 0x7fbab4101cd0 2024-09-12T02:49:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.149+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbaa4042030 con 0x7fbab4101cd0 2024-09-12T02:49:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.150+0000 7fbaaaffd640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 0x7fbaa00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.150+0000 7fbab9cde640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 0x7fbaa00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.150+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fbaa40efbe0 con 0x7fbab4101cd0 2024-09-12T02:49:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.151+0000 7fbab9cde640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 0x7fbaa00a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbab4199280 tx=0x7fbab000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.154+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbaa40bc170 con 0x7fbab4101cd0 2024-09-12T02:49:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.356+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fbab4062630 con 0x7fbaa00a2c00 2024-09-12T02:49:44.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:44 smithi046 ceph-mon[30951]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:44.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:44 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.357+0000 7fbaaaffd640 1 -- 172.21.15.46:0/2092963288 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fbab4062630 con 0x7fbaa00a2c00 2024-09-12T02:49:44.358 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-09-12T02:49:44.358 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-09-12T02:49:44.358 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 msgr2=0x7fbaa00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 0x7fbaa00a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbab4199280 tx=0x7fbab000b410 comp rx=0 tx=0).stop 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 msgr2=0x7fbab41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004770 tx=0x7fbaa40042f0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 shutdown_connections 2024-09-12T02:49:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbab4102690 0x7fbab419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbab4101cd0 0x7fbab41982e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fbaa00a2c00 0x7fbaa00a50c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 --2- 172.21.15.46:0/2092963288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab4100ad0 0x7fbab4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 >> 172.21.15.46:0/2092963288 conn(0x7fbab40fc280 msgr2=0x7fbab41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 shutdown_connections 2024-09-12T02:49:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:44.360+0000 7fbabbf42640 1 -- 172.21.15.46:0/2092963288 wait complete. 2024-09-12T02:49:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:44 smithi196 ceph-mon[34587]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:44 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:44 smithi145 ceph-mon[34674]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:44.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:44 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:45.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T02:49:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.859+0000 7f0d40a5a640 1 -- 172.21.15.46:0/2256513998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c101cd0 msgr2=0x7f0d3c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.859+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/2256513998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c102150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009900 tx=0x7f0d2402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.859+0000 7f0d40a5a640 1 -- 172.21.15.46:0/2256513998 shutdown_connections 2024-09-12T02:49:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/2256513998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c102690 0x7f0d3c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/2256513998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c102150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/2256513998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 0x7f0d3c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 -- 172.21.15.46:0/2256513998 >> 172.21.15.46:0/2256513998 conn(0x7f0d3c0fc280 msgr2=0x7f0d3c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 -- 172.21.15.46:0/2256513998 shutdown_connections 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 -- 172.21.15.46:0/2256513998 wait complete. 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.860+0000 7f0d40a5a640 1 Processor -- start 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 -- start start 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 0x7f0d3c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10c680 con 0x7f0d3c102690 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10c500 con 0x7f0d3c101cd0 2024-09-12T02:49:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d40a5a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10c800 con 0x7f0d3c100ad0 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55364/0 (socket says 172.21.15.46:55364) 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 -- 172.21.15.46:0/444862645 learned_addr learned my addr 172.21.15.46:0/444862645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d31d9b640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3a59c640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 0x7f0d3c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 -- 172.21.15.46:0/444862645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 msgr2=0x7f0d3c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 0x7f0d3c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 -- 172.21.15.46:0/444862645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c101cd0 msgr2=0x7f0d3c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 -- 172.21.15.46:0/444862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19fb30 con 0x7f0d3c102690 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.861+0000 7f0d3ad9d640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400eaa0 tx=0x7f0d3400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.862+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34004d20 con 0x7f0d3c102690 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.862+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19fe20 con 0x7f0d3c102690 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.862+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d34005020 con 0x7f0d3c102690 2024-09-12T02:49:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.862+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340116b0 con 0x7f0d3c102690 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.862+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a0350 con 0x7f0d3c102690 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.863+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c1a05e0 con 0x7f0d3c102690 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.864+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d340051d0 con 0x7f0d3c102690 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.865+0000 7f0d33fff640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 0x7f0d280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.865+0000 7f0d3a59c640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 0x7f0d280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.865+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f0d340cf520 con 0x7f0d3c102690 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.865+0000 7f0d3a59c640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 0x7f0d280a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c101b30 tx=0x7f0d2c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:45.869+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d34016030 con 0x7f0d3c102690 2024-09-12T02:49:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.053+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0d3c1a07b0 con 0x7f0d280a2bb0 2024-09-12T02:49:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.065+0000 7f0d33fff640 1 -- 172.21.15.46:0/444862645 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f0d3c1a07b0 con 0x7f0d280a2bb0 2024-09-12T02:49:46.066 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T02:49:46.066 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (2m) 0s ago 2m 15.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 9b2e540d37f8 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi145 running (2m) 2s ago 2m 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 6e329f6a256e 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi196 running (2m) 2s ago 2m 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 1626c637f9f7 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (2m) 0s ago 2m 16.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 334f22ba3c59 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi145 running (2m) 2s ago 2m 14.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 8a109ebc0bf3 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi196 running (2m) 2s ago 2m 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 6d931c7b2974 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (2m) 0s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 0f0a7aecd656 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi145 running (2m) 2s ago 2m 19.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 2fd53589e097 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi196 running (2m) 2s ago 2m 26.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 d3d11b84434f 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (2m) 0s ago 2m 16.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 10e27fc94aa4 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi145 running (2m) 2s ago 2m 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 6aec3e4d8a6b 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi196 running (2m) 2s ago 2m 20.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 237f921b7c9c 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (9m) 0s ago 9m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 f9e3e37a643a 2024-09-12T02:49:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi145 *:8443,8765 running (8m) 2s ago 8m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 d1fa9063c46f 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi196 *:8443,8765 running (8m) 2s ago 8m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 ed3aad2f73a8 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (9m) 0s ago 9m 56.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 3f65ff0b2fe5 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi145 running (8m) 2s ago 8m 42.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 be1aba72babc 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi196 running (8m) 2s ago 8m 43.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 58fda31218fb 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (7m) 0s ago 7m 48.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 444219c76550 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi145 running (7m) 2s ago 7m 62.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ffc4be8979ee 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi196 running (7m) 2s ago 7m 60.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ac4a1502d9e9 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (6m) 0s ago 6m 46.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f0905a7c0146 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi145 running (6m) 2s ago 6m 57.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 973823a80914 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi196 running (5m) 2s ago 5m 56.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fef5b2a50e7b 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (5m) 0s ago 5m 43.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 834a875ed174 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi145 running (4m) 2s ago 4m 60.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 e3b76555152b 2024-09-12T02:49:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi196 running (4m) 2s ago 4m 55.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 85f9ebd08e75 2024-09-12T02:49:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (3m) 0s ago 3m 44.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 4a4f4f243e96 2024-09-12T02:49:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi145 running (3m) 2s ago 3m 55.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 02b330a2c4e5 2024-09-12T02:49:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi196 running (3m) 2s ago 3m 43.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 622161dde0d9 2024-09-12T02:49:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 msgr2=0x7f0d280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 0x7f0d280a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c101b30 tx=0x7f0d2c009110 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 msgr2=0x7f0d3c19f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400eaa0 tx=0x7f0d3400eea0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 shutdown_connections 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d3c102690 0x7f0d3c19f4c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d3c101cd0 0x7f0d3c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d280a2bb0 0x7f0d280a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 --2- 172.21.15.46:0/444862645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d3c100ad0 0x7f0d3c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.067+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 >> 172.21.15.46:0/444862645 conn(0x7f0d3c0fc280 msgr2=0x7f0d3c069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.068+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 shutdown_connections 2024-09-12T02:49:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:46.068+0000 7f0d40a5a640 1 -- 172.21.15.46:0/444862645 wait complete. 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:46.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:46.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:46 smithi046 ceph-mon[30951]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:46.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:46 smithi196 ceph-mon[34587]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:46.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:46.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:46 smithi145 ceph-mon[34674]: from='mgr.14150 172.21.15.46:0/4018115766' entity='mgr.x' 2024-09-12T02:49:46.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T02:49:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:47.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:47 smithi046 ceph-mon[30951]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:47.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:47 smithi196 ceph-mon[34587]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 -- 172.21.15.46:0/3850337850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74101cd0 msgr2=0x7fdf74102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 --2- 172.21.15.46:0/3850337850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74101cd0 0x7fdf74102150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009900 tx=0x7fdf6402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 -- 172.21.15.46:0/3850337850 shutdown_connections 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 --2- 172.21.15.46:0/3850337850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74102690 0x7fdf74108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 --2- 172.21.15.46:0/3850337850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74101cd0 0x7fdf74102150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 --2- 172.21.15.46:0/3850337850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74100ad0 0x7fdf74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 -- 172.21.15.46:0/3850337850 >> 172.21.15.46:0/3850337850 conn(0x7fdf740fc280 msgr2=0x7fdf740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 -- 172.21.15.46:0/3850337850 shutdown_connections 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 -- 172.21.15.46:0/3850337850 wait complete. 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.536+0000 7fdf7c609640 1 Processor -- start 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 -- start start 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100ad0 0x7fdf74197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74101cd0 0x7fdf741983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c650 con 0x7fdf74102690 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c4d0 con 0x7fdf74101cd0 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7c609640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c7d0 con 0x7fdf74100ad0 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55386/0 (socket says 172.21.15.46:55386) 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 -- 172.21.15.46:0/1474376864 learned_addr learned my addr 172.21.15.46:0/1474376864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf79ba4640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74101cd0 0x7fdf741983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7a3a5640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100ad0 0x7fdf74197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 -- 172.21.15.46:0/1474376864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100ad0 msgr2=0x7fdf74197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100ad0 0x7fdf74197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 -- 172.21.15.46:0/1474376864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74101cd0 msgr2=0x7fdf741983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74101cd0 0x7fdf741983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 -- 172.21.15.46:0/1474376864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419fa90 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.537+0000 7fdf7aba6640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00cad0 tx=0x7fdf6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.538+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00ae20 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.538+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6c005020 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.538+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419fd80 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.538+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00f6b0 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.538+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a0290 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.539+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf741a0520 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.540+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf6c0051d0 con 0x7fdf74102690 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.540+0000 7fdf6b7fe640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 0x7fdf600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.541+0000 7fdf7a3a5640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 0x7fdf600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:47.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.541+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fdf6c0cf380 con 0x7fdf74102690 2024-09-12T02:49:47.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.541+0000 7fdf7a3a5640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 0x7fdf600a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005db0 tx=0x7fdf70005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:47.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.545+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf6c014030 con 0x7fdf74102690 2024-09-12T02:49:47.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:47 smithi145 ceph-mon[34674]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.728+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdf741a06f0 con 0x7fdf600a2c30 2024-09-12T02:49:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.735+0000 7fdf6b7fe640 1 -- 172.21.15.46:0/1474376864 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fdf741a06f0 con 0x7fdf600a2c30 2024-09-12T02:49:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T02:49:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 4s ago 2m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi145=b;smithi145=e;smithi145=h;smithi145=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-12T02:49:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 4s ago 8m smithi046=x;smithi145=y;smithi196=z;count:3 2024-09-12T02:49:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 4s ago 8m smithi046:172.21.15.46=a;smithi145:172.21.15.145=b;smithi196:172.21.15.196=c;count:3 2024-09-12T02:49:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 4s ago - 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 msgr2=0x7fdf600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 0x7fdf600a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005db0 tx=0x7fdf70005d40 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 msgr2=0x7fdf7419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00cad0 tx=0x7fdf6c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 shutdown_connections 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102690 0x7fdf7419f420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf74101cd0 0x7fdf741983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fdf600a2c30 0x7fdf600a50f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 --2- 172.21.15.46:0/1474376864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100ad0 0x7fdf74197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 >> 172.21.15.46:0/1474376864 conn(0x7fdf740fc280 msgr2=0x7fdf74069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 shutdown_connections 2024-09-12T02:49:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:47.737+0000 7fdf7c609640 1 -- 172.21.15.46:0/1474376864 wait complete. 2024-09-12T02:49:48.358 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:48 smithi046 ceph-mon[30951]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 111 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:48.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T02:49:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:48 smithi196 ceph-mon[34587]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 111 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:48 smithi145 ceph-mon[34674]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 111 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:48.646 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/3089317156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec102660 msgr2=0x7fc2ec104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/3089317156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec102660 0x7fc2ec104a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009900 tx=0x7fc2dc02a410 comp rx=0 tx=0).stop 2024-09-12T02:49:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/3089317156 shutdown_connections 2024-09-12T02:49:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/3089317156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/3089317156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec104f90 0x7fc2ec107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/3089317156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec102660 0x7fc2ec104a50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/3089317156 >> 172.21.15.46:0/3089317156 conn(0x7fc2ec0fc280 msgr2=0x7fc2ec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/3089317156 shutdown_connections 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.186+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/3089317156 wait complete. 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 Processor -- start 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 -- start start 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec102660 0x7fc2ec19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec104f90 0x7fc2ec19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec10d790 con 0x7fc2ec1078c0 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec10d610 con 0x7fc2ec102660 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2f1fc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec10d910 con 0x7fc2ec104f90 2024-09-12T02:49:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55398/0 (socket says 172.21.15.46:55398) 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2ebfff640 1 -- 172.21.15.46:0/2067530159 learned_addr learned my addr 172.21.15.46:0/2067530159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2eaffd640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec104f90 0x7fc2ec19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.187+0000 7fc2eb7fe640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec102660 0x7fc2ec19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 -- 172.21.15.46:0/2067530159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec104f90 msgr2=0x7fc2ec19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec104f90 0x7fc2ec19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 -- 172.21.15.46:0/2067530159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec102660 msgr2=0x7fc2ec19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec102660 0x7fc2ec19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 -- 172.21.15.46:0/2067530159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec1a1c90 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2ebfff640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400cce0 tx=0x7fc2e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400f450 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2e400faa0 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec1a1f80 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4018a80 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.188+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec1a2460 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.189+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec109410 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.190+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2e400fc50 con 0x7fc2ec1078c0 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.191+0000 7fc2e8ff9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 0x7fc2d40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.191+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fc2e40cf760 con 0x7fc2ec1078c0 2024-09-12T02:49:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.191+0000 7fc2eb7fe640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 0x7fc2d40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.191+0000 7fc2eb7fe640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 0x7fc2d40a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000c00 tx=0x7fc2dc005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.195+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2e4014030 con 0x7fc2ec1078c0 2024-09-12T02:49:49.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.378+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc2ec062630 con 0x7fc2d40a2bb0 2024-09-12T02:49:49.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:49 smithi046 ceph-mon[30951]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.380+0000 7fc2e8ff9640 1 -- 172.21.15.46:0/2067530159 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc2ec062630 con 0x7fc2d40a2bb0 2024-09-12T02:49:49.381 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-09-12T02:49:49.381 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-09-12T02:49:49.381 INFO:teuthology.orchestra.run.smithi046.stdout:smithi145 172.21.15.145 2024-09-12T02:49:49.381 INFO:teuthology.orchestra.run.smithi046.stdout:smithi196 172.21.15.196 2024-09-12T02:49:49.381 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-09-12T02:49:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 msgr2=0x7fc2d40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 0x7fc2d40a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000c00 tx=0x7fc2dc005e40 comp rx=0 tx=0).stop 2024-09-12T02:49:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 msgr2=0x7fc2ec1a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400cce0 tx=0x7fc2e400a620 comp rx=0 tx=0).stop 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 shutdown_connections 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ec1078c0 0x7fc2ec1a1620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc2d40a2bb0 0x7fc2d40a5070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ec104f90 0x7fc2ec19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 --2- 172.21.15.46:0/2067530159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ec102660 0x7fc2ec19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 >> 172.21.15.46:0/2067530159 conn(0x7fc2ec0fc280 msgr2=0x7fc2ec101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.382+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 shutdown_connections 2024-09-12T02:49:49.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:49.383+0000 7fc2f1fc9640 1 -- 172.21.15.46:0/2067530159 wait complete. 2024-09-12T02:49:49.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:49 smithi196 ceph-mon[34587]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:49.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:49 smithi145 ceph-mon[34674]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:50.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T02:49:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:50.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:50 smithi046 ceph-mon[30951]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:50 smithi196 ceph-mon[34587]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:50 smithi145 ceph-mon[34674]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 -- 172.21.15.46:0/1188064965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 msgr2=0x7f6d44102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 --2- 172.21.15.46:0/1188064965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44102150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009900 tx=0x7f6d3802a410 comp rx=0 tx=0).stop 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 -- 172.21.15.46:0/1188064965 shutdown_connections 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 --2- 172.21.15.46:0/1188064965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d44108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 --2- 172.21.15.46:0/1188064965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44102150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 --2- 172.21.15.46:0/1188064965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 0x7f6d44100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 -- 172.21.15.46:0/1188064965 >> 172.21.15.46:0/1188064965 conn(0x7f6d440fc280 msgr2=0x7f6d440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.796+0000 7f6d49941640 1 -- 172.21.15.46:0/1188064965 shutdown_connections 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 -- 172.21.15.46:0/1188064965 wait complete. 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 Processor -- start 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 -- start start 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 0x7f6d44197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d4419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4410c680 con 0x7f6d44101cd0 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4410c500 con 0x7f6d44102690 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.797+0000 7f6d49941640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4410c800 con 0x7f6d44100ad0 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55410/0 (socket says 172.21.15.46:55410) 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 -- 172.21.15.46:0/335534081 learned_addr learned my addr 172.21.15.46:0/335534081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d42ffd640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 0x7f6d44197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d437fe640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d4419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 -- 172.21.15.46:0/335534081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 msgr2=0x7f6d44197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 0x7f6d44197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 -- 172.21.15.46:0/335534081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 msgr2=0x7f6d4419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d4419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 -- 172.21.15.46:0/335534081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419fb30 con 0x7f6d44101cd0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d437fe640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d4419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d427fc640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d38004770 tx=0x7f6d380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3803f030 con 0x7f6d44101cd0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d38003cb0 con 0x7f6d44101cd0 2024-09-12T02:49:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419fdc0 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38035070 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.798+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a02e0 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.799+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d441a0510 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.801+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d38042030 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.801+0000 7f6d4893f640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 0x7f6d2c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.801+0000 7f6d42ffd640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 0x7f6d2c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.801+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f6d380ef910 con 0x7f6d44101cd0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.802+0000 7f6d42ffd640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 0x7f6d2c0a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005fd0 tx=0x7f6d3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:50.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.806+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d380bc4f0 con 0x7f6d44101cd0 2024-09-12T02:49:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.988+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 --> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6d4406a6f0 con 0x7f6d2c0a2c30 2024-09-12T02:49:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.990+0000 7f6d4893f640 1 -- 172.21.15.46:0/335534081 <== mgr.14150 v2:172.21.15.46:6800/3748867549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6d4406a6f0 con 0x7f6d2c0a2c30 2024-09-12T02:49:50.991 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T02:49:50.991 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:49:50.991 INFO:teuthology.orchestra.run.smithi046.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:49:50.991 INFO:teuthology.orchestra.run.smithi046.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 msgr2=0x7f6d2c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 0x7f6d2c0a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005fd0 tx=0x7f6d3400b390 comp rx=0 tx=0).stop 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 msgr2=0x7f6d44198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d38004770 tx=0x7f6d380042f0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 shutdown_connections 2024-09-12T02:49:50.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d44102690 0x7f6d4419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d44101cd0 0x7f6d44198440 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6d2c0a2c30 0x7f6d2c0a50f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 --2- 172.21.15.46:0/335534081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44100ad0 0x7f6d44197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 >> 172.21.15.46:0/335534081 conn(0x7f6d440fc280 msgr2=0x7f6d44069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 shutdown_connections 2024-09-12T02:49:50.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:50.994+0000 7f6d49941640 1 -- 172.21.15.46:0/335534081 wait complete. 2024-09-12T02:49:51.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:51 smithi046 ceph-mon[30951]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:51.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:51 smithi196 ceph-mon[34587]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:51.511 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:51 smithi145 ceph-mon[34674]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:51.754 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:49:51.765 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:49:51.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T02:49:52.023 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:52.422 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:52 smithi046 ceph-mon[30951]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:52.422 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:52 smithi046 ceph-mon[30951]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:52 smithi196 ceph-mon[34587]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:52 smithi196 ceph-mon[34587]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 -- 172.21.15.46:0/1076516969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80102e00 msgr2=0x7f0d80103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 --2- 172.21.15.46:0/1076516969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80102e00 0x7f0d80103280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d7402a410 comp rx=0 tx=0).stop 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 -- 172.21.15.46:0/1076516969 shutdown_connections 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 --2- 172.21.15.46:0/1076516969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d801037c0 0x7f0d8010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 --2- 172.21.15.46:0/1076516969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80102e00 0x7f0d80103280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 --2- 172.21.15.46:0/1076516969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d80101c00 0x7f0d80102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 -- 172.21.15.46:0/1076516969 >> 172.21.15.46:0/1076516969 conn(0x7f0d800fd390 msgr2=0x7f0d800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 -- 172.21.15.46:0/1076516969 shutdown_connections 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.505+0000 7f0d86614640 1 -- 172.21.15.46:0/1076516969 wait complete. 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 Processor -- start 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 -- start start 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d80102e00 0x7f0d800745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 0x7f0d80074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d780 con 0x7f0d80101c00 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d600 con 0x7f0d80102e00 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d86614640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d900 con 0x7f0d801037c0 2024-09-12T02:49:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55434/0 (socket says 172.21.15.46:55434) 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.506+0000 7f0d7ffff640 1 -- 172.21.15.46:0/147499013 learned_addr learned my addr 172.21.15.46:0/147499013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7f7fe640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d80102e00 0x7f0d800745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d84bb1640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 0x7f0d80074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 -- 172.21.15.46:0/147499013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 msgr2=0x7f0d80074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 0x7f0d80074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 -- 172.21.15.46:0/147499013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d80102e00 msgr2=0x7f0d800745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d80102e00 0x7f0d800745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 -- 172.21.15.46:0/147499013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d800753e0 con 0x7f0d80101c00 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d84bb1640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 0x7f0d80074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7ffff640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c970 tx=0x7f0d7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004220 con 0x7f0d80101c00 2024-09-12T02:49:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d700043c0 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a1fa0 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004a00 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.507+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a2340 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.508+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d80108ff0 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.509+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d700079b0 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.510+0000 7f0d7d7fa640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 0x7f0d680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.510+0000 7f0d7f7fe640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 0x7f0d680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.510+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f0d700cf1f0 con 0x7f0d80101c00 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.511+0000 7f0d7f7fe640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 0x7f0d680a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d740314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.515+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d70014030 con 0x7f0d80101c00 2024-09-12T02:49:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:52 smithi145 ceph-mon[34674]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:49:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:52 smithi145 ceph-mon[34674]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.729+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0d80062e90 con 0x7f0d80101c00 2024-09-12T02:49:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.730+0000 7f0d7d7fa640 1 -- 172.21.15.46:0/147499013 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3096 (secure 0 0 0) 0x7f0d7009bdd0 con 0x7f0d80101c00 2024-09-12T02:49:52.730 INFO:teuthology.orchestra.run.smithi046.stdout:e24 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-12T02:48:22:224987+0000 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 24 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-12T02:47:41.799879+0000 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-12T02:48:21.888091+0000 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-12T02:49:52.731 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 3 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24478,1=24476,2=24460} 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-12T02:49:52.732 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3,4] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 24478 members: 24478,24476,24460 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{0:24478} state up:active seq 7 addr [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/3561512892,v1:172.21.15.196:6835/3561512892] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{1:24476} state up:active seq 13 addr [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{1:24446} state up:standby-replay seq 1 addr [v2:172.21.15.145:6836/2556040092,v1:172.21.15.145:6837/2556040092] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{2:24460} state up:active seq 17 addr [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/1048929159,v1:172.21.15.196:6833/1048929159] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14514} state up:standby seq 1 addr [v2:172.21.15.46:6834/2542241089,v1:172.21.15.46:6835/2542241089] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14526} state up:standby seq 1 addr [v2:172.21.15.46:6836/4040861117,v1:172.21.15.46:6837/4040861117] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{-1:14568} state up:standby seq 1 addr [v2:172.21.15.46:6838/2433411125,v1:172.21.15.46:6839/2433411125] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{-1:14574} state up:standby seq 1 addr [v2:172.21.15.46:6840/1897348158,v1:172.21.15.46:6841/1897348158] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{-1:24404} state up:standby seq 1 addr [v2:172.21.15.145:6832/1820698401,v1:172.21.15.145:6833/1820698401] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{-1:24425} state up:standby seq 1 addr [v2:172.21.15.145:6834/198433993,v1:172.21.15.145:6835/198433993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 msgr2=0x7f0d680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 0x7f0d680a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d740314a0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 msgr2=0x7f0d80075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c970 tx=0x7f0d7000ce30 comp rx=0 tx=0).stop 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 shutdown_connections 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f0d680a2b80 0x7f0d680a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d801037c0 0x7f0d80074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d80102e00 0x7f0d800745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 --2- 172.21.15.46:0/147499013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d80101c00 0x7f0d80075f90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 >> 172.21.15.46:0/147499013 conn(0x7f0d800fd390 msgr2=0x7f0d801061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 shutdown_connections 2024-09-12T02:49:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:52.732+0000 7f0d86614640 1 -- 172.21.15.46:0/147499013 wait complete. 2024-09-12T02:49:53.420 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:53 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/147499013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:49:53.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T02:49:53.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:53 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/147499013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:49:53.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:53 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/147499013' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:49:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:54.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 -- 172.21.15.46:0/3136355954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c102660 msgr2=0x7fd87c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 --2- 172.21.15.46:0/3136355954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c102660 0x7fd87c104a50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009900 tx=0x7fd86c02a410 comp rx=0 tx=0).stop 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 -- 172.21.15.46:0/3136355954 shutdown_connections 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 --2- 172.21.15.46:0/3136355954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 0x7fd87c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 --2- 172.21.15.46:0/3136355954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c104f90 0x7fd87c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 --2- 172.21.15.46:0/3136355954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c102660 0x7fd87c104a50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 -- 172.21.15.46:0/3136355954 >> 172.21.15.46:0/3136355954 conn(0x7fd87c0fc280 msgr2=0x7fd87c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 -- 172.21.15.46:0/3136355954 shutdown_connections 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.162+0000 7fd882379640 1 -- 172.21.15.46:0/3136355954 wait complete. 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 Processor -- start 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 -- start start 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c102660 0x7fd87c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:54.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 0x7fd87c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d720 con 0x7fd87c104f90 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d5a0 con 0x7fd87c1078c0 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.163+0000 7fd882379640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d8a0 con 0x7fd87c102660 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55446/0 (socket says 172.21.15.46:55446) 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 -- 172.21.15.46:0/3975515266 learned_addr learned my addr 172.21.15.46:0/3975515266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87bfff640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c102660 0x7fd87c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd880916640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 0x7fd87c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 -- 172.21.15.46:0/3975515266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c102660 msgr2=0x7fd87c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c102660 0x7fd87c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 -- 172.21.15.46:0/3975515266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 msgr2=0x7fd87c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 0x7fd87c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 -- 172.21.15.46:0/3975515266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c101410 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd87b7fe640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd87000d770 tx=0x7fd87000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870017070 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd870009c30 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a1fc0 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000f620 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.164+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1a2470 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.165+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c0fdd70 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.167+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd87001a070 con 0x7fd87c104f90 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.167+0000 7fd8797fa640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 0x7fd8640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.167+0000 7fd87bfff640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 0x7fd8640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.167+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fd8700ced80 con 0x7fd87c104f90 2024-09-12T02:49:54.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.168+0000 7fd87bfff640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 0x7fd8640a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006010 tx=0x7fd86c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:54.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.170+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd870009de0 con 0x7fd87c104f90 2024-09-12T02:49:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.348+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd87c02d7c0 con 0x7fd87c104f90 2024-09-12T02:49:54.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:54 smithi046 ceph-mon[30951]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.349+0000 7fd8797fa640 1 -- 172.21.15.46:0/3975515266 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5613 (secure 0 0 0) 0x7fd87009b310 con 0x7fd87c104f90 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 92 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-12T02:39:49.303412+0000 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-12T02:49:38.126353+0000 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 27 2024-09-12T02:49:54.350 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/836973982,v1:172.21.15.46:6803/836973982] [v2:172.21.15.46:6804/836973982,v1:172.21.15.46:6805/836973982] exists,up aec0a35b-0993-4783-ae91-001115bfc027 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/1956814217,v1:172.21.15.145:6801/1956814217] [v2:172.21.15.145:6802/1956814217,v1:172.21.15.145:6803/1956814217] exists,up 701e099d-ad1c-45cd-b9e3-8164c0daa034 2024-09-12T02:49:54.351 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/3433225191,v1:172.21.15.196:6801/3433225191] [v2:172.21.15.196:6802/3433225191,v1:172.21.15.196:6803/3433225191] exists,up ab049462-5f42-4b64-8536-edca415271ff 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/3782480680,v1:172.21.15.46:6811/3782480680] [v2:172.21.15.46:6812/3782480680,v1:172.21.15.46:6813/3782480680] exists,up 6ad0adf4-ee5f-4d68-8b92-27397454b3ee 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/473009837,v1:172.21.15.145:6809/473009837] [v2:172.21.15.145:6810/473009837,v1:172.21.15.145:6811/473009837] exists,up 62730f75-0e03-48fa-9cd3-d04a9e1d0046 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 38 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/859658761,v1:172.21.15.196:6809/859658761] [v2:172.21.15.196:6810/859658761,v1:172.21.15.196:6811/859658761] exists,up 50ec1203-dfd3-402e-ae2f-e1c914c74c02 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 45 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/2825700382,v1:172.21.15.46:6819/2825700382] [v2:172.21.15.46:6820/2825700382,v1:172.21.15.46:6821/2825700382] exists,up 499ded47-5f1d-433f-8a32-c75b6b64106a 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/1917623570,v1:172.21.15.145:6817/1917623570] [v2:172.21.15.145:6818/1917623570,v1:172.21.15.145:6819/1917623570] exists,up 7de3c410-22b1-47ea-94ef-93b756a84fc8 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 57 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/724845953,v1:172.21.15.196:6817/724845953] [v2:172.21.15.196:6818/724845953,v1:172.21.15.196:6819/724845953] exists,up 040d8985-a30f-4bdd-a469-0be6e29b98a8 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 64 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/182095023,v1:172.21.15.46:6827/182095023] [v2:172.21.15.46:6828/182095023,v1:172.21.15.46:6829/182095023] exists,up 5445403e-05c9-4020-b773-336395f26213 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/140848472,v1:172.21.15.145:6825/140848472] [v2:172.21.15.145:6826/140848472,v1:172.21.15.145:6827/140848472] exists,up f5fbcc26-7505-434d-b947-79310be95234 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 76 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/3261621208,v1:172.21.15.196:6825/3261621208] [v2:172.21.15.196:6826/3261621208,v1:172.21.15.196:6827/3261621208] exists,up f2a6c3d9-b9e3-4985-b4b1-e1ac67272126 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.a [2,11] 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.13 [2,11] 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1 [4,9] 2024-09-12T02:49:54.352 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3 [4,9] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1a [5,6] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.22 [3,9] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.27 [4,8] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.6 [9,8] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.b [9,8] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.e [5,8] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.16 [9,8] 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2936988098 expires 2024-09-13T02:40:36.157891+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/4279295285 expires 2024-09-13T02:40:36.157891+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2423476506 expires 2024-09-13T02:40:36.157891+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1895867717 expires 2024-09-13T02:40:11.855373+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2351168704 expires 2024-09-13T02:40:36.157891+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/4279295285 expires 2024-09-13T02:40:36.157891+0000 2024-09-12T02:49:54.353 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/4229481015 expires 2024-09-13T02:40:11.855373+0000 2024-09-12T02:49:54.354 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/4229481015 expires 2024-09-13T02:40:11.855373+0000 2024-09-12T02:49:54.354 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/829485206 expires 2024-09-13T02:40:11.855373+0000 2024-09-12T02:49:54.354 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/791127312 expires 2024-09-13T02:40:11.855373+0000 2024-09-12T02:49:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.351+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 msgr2=0x7fd8640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.351+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 0x7fd8640a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006010 tx=0x7fd86c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.351+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 msgr2=0x7fd87c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.351+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd87000d770 tx=0x7fd87000dc30 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 shutdown_connections 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd87c1078c0 0x7fd87c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87c104f90 0x7fd87c100610 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fd8640a2b80 0x7fd8640a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 --2- 172.21.15.46:0/3975515266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd87c102660 0x7fd87c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 >> 172.21.15.46:0/3975515266 conn(0x7fd87c0fc280 msgr2=0x7fd87c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 shutdown_connections 2024-09-12T02:49:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:54.352+0000 7fd882379640 1 -- 172.21.15.46:0/3975515266 wait complete. 2024-09-12T02:49:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:54 smithi196 ceph-mon[34587]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-12T02:49:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:54 smithi145 ceph-mon[34674]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-12T02:49:55.069 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T02:49:55.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:49:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:55.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:55 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3975515266' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:49:55.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:55 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3975515266' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:49:55.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:55 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3975515266' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:49:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- 172.21.15.46:0/1830683563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 msgr2=0x7ffba4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 --2- 172.21.15.46:0/1830683563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba4101200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009900 tx=0x7ffba002a480 comp rx=0 tx=0).stop 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- 172.21.15.46:0/1830683563 shutdown_connections 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 --2- 172.21.15.46:0/1830683563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102ef0 0x7ffba410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 --2- 172.21.15.46:0/1830683563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102570 0x7ffba41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 --2- 172.21.15.46:0/1830683563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba4101200 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- 172.21.15.46:0/1830683563 >> 172.21.15.46:0/1830683563 conn(0x7ffba40fc7d0 msgr2=0x7ffba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- 172.21.15.46:0/1830683563 shutdown_connections 2024-09-12T02:49:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- 172.21.15.46:0/1830683563 wait complete. 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 Processor -- start 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.906+0000 7ffbab286640 1 -- start start 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102570 0x7ffba419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 0x7ffba419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4110e50 con 0x7ffba4100e20 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4110cd0 con 0x7ffba4102570 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffbab286640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4110fd0 con 0x7ffba4102ef0 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40612/0 (socket says 172.21.15.46:40612) 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 -- 172.21.15.46:0/1888264975 learned_addr learned my addr 172.21.15.46:0/1888264975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba8821640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102570 0x7ffba419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9823640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 0x7ffba419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 -- 172.21.15.46:0/1888264975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 msgr2=0x7ffba419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 0x7ffba419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 -- 172.21.15.46:0/1888264975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102570 msgr2=0x7ffba419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102570 0x7ffba419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 -- 172.21.15.46:0/1888264975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba419f570 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9823640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 0x7ffba419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.907+0000 7ffba9022640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006010 tx=0x7ffba00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.908+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba003f070 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.908+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba419f800 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.908+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba0003aa0 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.908+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0035030 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.908+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba41a7140 con 0x7ffba4100e20 2024-09-12T02:49:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.909+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba41021b0 con 0x7ffba4100e20 2024-09-12T02:49:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.910+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffba0042070 con 0x7ffba4100e20 2024-09-12T02:49:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.910+0000 7ffb9a7fc640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 0x7ffb900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.911+0000 7ffba8821640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 0x7ffb900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.911+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7ffba00f0340 con 0x7ffba4100e20 2024-09-12T02:49:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.911+0000 7ffba8821640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 0x7ffb900a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffba4104c00 tx=0x7ffb94009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:55.914+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffba00bcfc0 con 0x7ffba4100e20 2024-09-12T02:49:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.125+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba41a7310 con 0x7ffba4100e20 2024-09-12T02:49:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.125+0000 7ffb9a7fc640 1 -- 172.21.15.46:0/1888264975 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7ffba00bc8c0 con 0x7ffba4100e20 2024-09-12T02:49:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:56.126 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:49:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 msgr2=0x7ffb900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 0x7ffb900a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffba4104c00 tx=0x7ffb94009110 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 msgr2=0x7ffba419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006010 tx=0x7ffba00042f0 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 shutdown_connections 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ffb900a2b80 0x7ffb900a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4102ef0 0x7ffba419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffba4102570 0x7ffba419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 --2- 172.21.15.46:0/1888264975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffba4100e20 0x7ffba419a5c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 >> 172.21.15.46:0/1888264975 conn(0x7ffba40fc7d0 msgr2=0x7ffba406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 shutdown_connections 2024-09-12T02:49:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:56.128+0000 7ffbab286640 1 -- 172.21.15.46:0/1888264975 wait complete. 2024-09-12T02:49:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:56 smithi196 ceph-mon[34587]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:56 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1888264975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:56.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:56 smithi046 ceph-mon[30951]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:56.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:56 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1888264975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:56 smithi145 ceph-mon[34674]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:49:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:56 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1888264975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:56.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:49:57.096 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 -- 172.21.15.46:0/2636019795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 msgr2=0x7f1604106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 --2- 172.21.15.46:0/2636019795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f1604106e60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009930 tx=0x7f15f402a420 comp rx=0 tx=0).stop 2024-09-12T02:49:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 -- 172.21.15.46:0/2636019795 shutdown_connections 2024-09-12T02:49:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 --2- 172.21.15.46:0/2636019795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 0x7f160410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 --2- 172.21.15.46:0/2636019795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 0x7f1604100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 --2- 172.21.15.46:0/2636019795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f1604106e60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 -- 172.21.15.46:0/2636019795 >> 172.21.15.46:0/2636019795 conn(0x7f16040fc7a0 msgr2=0x7f16040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 -- 172.21.15.46:0/2636019795 shutdown_connections 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.596+0000 7f1609613640 1 -- 172.21.15.46:0/2636019795 wait complete. 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 Processor -- start 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 -- start start 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 0x7f160410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 0x7f160410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1604111e50 con 0x7f1604106a80 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1604111cd0 con 0x7f1604100a80 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f1609613640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1604111fd0 con 0x7f1604101420 2024-09-12T02:49:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f16037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f16037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40622/0 (socket says 172.21.15.46:40622) 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.597+0000 7f16037fe640 1 -- 172.21.15.46:0/529664982 learned_addr learned my addr 172.21.15.46:0/529664982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1602ffd640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 0x7f160410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f15fbfff640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 0x7f160410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 -- 172.21.15.46:0/529664982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 msgr2=0x7f160410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 0x7f160410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 -- 172.21.15.46:0/529664982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 msgr2=0x7f160410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 0x7f160410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 -- 172.21.15.46:0/529664982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041a6850 con 0x7f1604106a80 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f16037fe640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00eaa0 tx=0x7f15fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00cdb0 con 0x7f1604106a80 2024-09-12T02:49:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15fc005020 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041a6b40 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0116d0 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.598+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041a7080 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.599+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1604100ee0 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.601+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15fc0051d0 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.601+0000 7f1600ff9640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 0x7f15f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.601+0000 7f1602ffd640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 0x7f15f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.601+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f15fc0cf8c0 con 0x7f1604106a80 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.602+0000 7f1602ffd640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 0x7f15f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009900 tx=0x7f15f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:57.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.604+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15fc016030 con 0x7f1604106a80 2024-09-12T02:49:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.817+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f160410f910 con 0x7f1604106a80 2024-09-12T02:49:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.818+0000 7f1600ff9640 1 -- 172.21.15.46:0/529664982 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11924 (secure 0 0 0) 0x7f160410f910 con 0x7f1604106a80 2024-09-12T02:49:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:57.819 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-12T02:48:22:224987+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T02:48:21.888091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T02:49:57.819 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 msgr2=0x7f15f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 0x7f15f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009900 tx=0x7f15f4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 msgr2=0x7f160410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00eaa0 tx=0x7f15fc00ef60 comp rx=0 tx=0).stop 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 shutdown_connections 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1604106a80 0x7f160410ee60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f15f00a2b80 0x7f15f00a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1604101420 0x7f160410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 --2- 172.21.15.46:0/529664982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1604100a80 0x7f160410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 >> 172.21.15.46:0/529664982 conn(0x7f16040fc7a0 msgr2=0x7f16040fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 shutdown_connections 2024-09-12T02:49:57.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:57.820+0000 7f1609613640 1 -- 172.21.15.46:0/529664982 wait complete. 2024-09-12T02:49:58.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:58 smithi196 ceph-mon[34587]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:58.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:49:58 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/529664982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:58.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:58 smithi046 ceph-mon[30951]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:58.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:49:58 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/529664982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:58.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:49:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:58 smithi145 ceph-mon[34674]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:49:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:49:58 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/529664982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:49:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:59.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 -- 172.21.15.46:0/2761978556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4103660 msgr2=0x7fc8f410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:59.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2761978556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4103660 0x7fc8f410bb50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009900 tx=0x7fc8dc02a480 comp rx=0 tx=0).stop 2024-09-12T02:49:59.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 -- 172.21.15.46:0/2761978556 shutdown_connections 2024-09-12T02:49:59.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2761978556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 0x7fc8f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2761978556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4103660 0x7fc8f410bb50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2761978556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4102d40 0x7fc8f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 -- 172.21.15.46:0/2761978556 >> 172.21.15.46:0/2761978556 conn(0x7fc8f40fc7a0 msgr2=0x7fc8f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 -- 172.21.15.46:0/2761978556 shutdown_connections 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.289+0000 7fc8f920c640 1 -- 172.21.15.46:0/2761978556 wait complete. 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 Processor -- start 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 -- start start 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4103660 0x7fc8f4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 0x7fc8f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111e60 con 0x7fc8f4102d40 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111ce0 con 0x7fc8f4103660 2024-09-12T02:49:59.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f920c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111fe0 con 0x7fc8f410c170 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40632/0 (socket says 172.21.15.46:40632) 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 -- 172.21.15.46:0/2160879334 learned_addr learned my addr 172.21.15.46:0/2160879334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f359e640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 0x7fc8f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8ebfff640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4103660 0x7fc8f4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 -- 172.21.15.46:0/2160879334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 msgr2=0x7fc8f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 0x7fc8f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 -- 172.21.15.46:0/2160879334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4103660 msgr2=0x7fc8f4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4103660 0x7fc8f4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.290+0000 7fc8f2d9d640 1 -- 172.21.15.46:0/2160879334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419d300 con 0x7fc8f4102d40 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f2d9d640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c920 tx=0x7fc8e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007c70 con 0x7fc8f4102d40 2024-09-12T02:49:59.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e4007e10 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419d5f0 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4018820 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.291+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a4ed0 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.292+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4103120 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.293+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8e40189c0 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.294+0000 7fc8f0d99640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 0x7fc8e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.294+0000 7fc8ebfff640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 0x7fc8e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.294+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7fc8e40cf440 con 0x7fc8f4102d40 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.294+0000 7fc8ebfff640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 0x7fc8e00a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc031440 tx=0x7fc8dc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.298+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8e4014030 con 0x7fc8f4102d40 2024-09-12T02:49:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.475+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8f4106c50 con 0x7fc8f4102d40 2024-09-12T02:49:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.477+0000 7fc8f0d99640 1 -- 172.21.15.46:0/2160879334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24703 (secure 0 0 0) 0x7fc8e409bfc0 con 0x7fc8f4102d40 2024-09-12T02:49:59.478 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T02:49:59.478 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":92,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T02:49:38.126353+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:49:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 msgr2=0x7fc8e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 0x7fc8e00a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc031440 tx=0x7fc8dc005e80 comp rx=0 tx=0).stop 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 msgr2=0x7fc8f4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c920 tx=0x7fc8e400cde0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 shutdown_connections 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7fc8e00a2c00 0x7fc8e00a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c170 0x7fc8f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4103660 0x7fc8f4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 --2- 172.21.15.46:0/2160879334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4102d40 0x7fc8f4198350 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 >> 172.21.15.46:0/2160879334 conn(0x7fc8f40fc7a0 msgr2=0x7fc8f40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 shutdown_connections 2024-09-12T02:49:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:49:59.479+0000 7fc8f920c640 1 -- 172.21.15.46:0/2160879334 wait complete. 2024-09-12T02:50:00.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:47:41.799879+0000', 'modified': '2024-09-12T02:48:21.888091+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24476, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1048929159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1048929159}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1048929159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/2556040092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 2556040092}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 2556040092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/3561512892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3561512892}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3561512892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6837/756659635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 756659635}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 756659635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.145:6839/1761070564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 1761070564}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 1761070564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6839/1262634008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1262634008}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1262634008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24478, 24476, 24460]} 2024-09-12T02:50:00.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T02:50:00.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:50:00.458 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:00 smithi196 ceph-mon[34587]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:50:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:00 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2160879334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:50:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:00 smithi196 ceph-mon[34587]: overall HEALTH_OK 2024-09-12T02:50:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:00 smithi046 ceph-mon[30951]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:50:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:00 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2160879334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:50:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:00 smithi046 ceph-mon[30951]: overall HEALTH_OK 2024-09-12T02:50:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:00 smithi145 ceph-mon[34674]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:50:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:00 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2160879334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:50:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:00 smithi145 ceph-mon[34674]: overall HEALTH_OK 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 -- 172.21.15.46:0/4204573838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 msgr2=0x7ff87810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 --2- 172.21.15.46:0/4204573838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87810ea90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ca0 tx=0x7ff87002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 -- 172.21.15.46:0/4204573838 shutdown_connections 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 --2- 172.21.15.46:0/4204573838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87810ea90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 --2- 172.21.15.46:0/4204573838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878102790 0x7ff8781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 --2- 172.21.15.46:0/4204573838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878101de0 0x7ff8781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 -- 172.21.15.46:0/4204573838 >> 172.21.15.46:0/4204573838 conn(0x7ff8780fd8e0 msgr2=0x7ff8780ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 -- 172.21.15.46:0/4204573838 shutdown_connections 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 -- 172.21.15.46:0/4204573838 wait complete. 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.984+0000 7ff87c938640 1 Processor -- start 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 -- start start 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 0x7ff8781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 0x7ff878198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff878111e70 con 0x7ff878106f10 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff878111cf0 con 0x7ff878101de0 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87c938640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff878111ff0 con 0x7ff878102790 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40654/0 (socket says 172.21.15.46:40654) 2024-09-12T02:50:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 -- 172.21.15.46:0/1046970385 learned_addr learned my addr 172.21.15.46:0/1046970385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff875d9b640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 0x7ff878198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87659c640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 0x7ff8781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 -- 172.21.15.46:0/1046970385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 msgr2=0x7ff878198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 0x7ff878198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 -- 172.21.15.46:0/1046970385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 msgr2=0x7ff8781982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 0x7ff8781982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 -- 172.21.15.46:0/1046970385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87819d2a0 con 0x7ff878106f10 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff875d9b640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 0x7ff878198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff87659c640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 0x7ff8781982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.985+0000 7ff876d9d640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff870031040 tx=0x7ff870003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.986+0000 7ff86f7fe640 1 -- 172.21.15.46:0/1046970385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870004040 con 0x7ff878106f10 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.986+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87819d530 con 0x7ff878106f10 2024-09-12T02:50:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.986+0000 7ff86f7fe640 1 -- 172.21.15.46:0/1046970385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff870004690 con 0x7ff878106f10 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.986+0000 7ff86f7fe640 1 -- 172.21.15.46:0/1046970385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003b410 con 0x7ff878106f10 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.986+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a4e70 con 0x7ff878106f10 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.988+0000 7ff86f7fe640 1 -- 172.21.15.46:0/1046970385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff8700041e0 con 0x7ff878106f10 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.988+0000 7ff87ed11740 1 Processor -- start 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.988+0000 7ff87ed11740 1 -- start start 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c3e525090 con 0x561c3e5bf1d0 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c3e4cca70 con 0x561c3e5cdfc0 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87ed11740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c3e521050 con 0x561c3e5be8b0 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff86f7fe640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 0x7ff8640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40668/0 (socket says 172.21.15.46:40668) 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 -- 172.21.15.46:0/1758898267 learned_addr learned my addr 172.21.15.46:0/1758898267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff875d9b640 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff876d9d640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 0x7ff8640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff875d9b640 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 -- 172.21.15.46:0/1758898267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 msgr2=0x561c3e5bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 -- 172.21.15.46:0/1758898267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 msgr2=0x561c3e5d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff86f7fe640 1 -- 172.21.15.46:0/1046970385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7ff8700f0c50 con 0x7ff878106f10 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 -- 172.21.15.46:0/1758898267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3e4d9130 con 0x561c3e5bf1d0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.989+0000 7ff87659c640 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff86800cca0 tx=0x7ff86800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d14889be9d8b72b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:00.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff876d9d640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 0x7ff8640a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ee740 tx=0x7ff8700e7cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff86cff9640 1 -- 172.21.15.46:0/1758898267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868017dd0 con 0x561c3e5bf1d0 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff86cff9640 1 -- 172.21.15.46:0/1758898267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8680054d0 con 0x561c3e5bf1d0 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff86cff9640 1 -- 172.21.15.46:0/1758898267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868021430 con 0x561c3e5bf1d0 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff87ed11740 1 -- 172.21.15.46:0/1758898267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 msgr2=0x561c3e5cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.990+0000 7ff87ed11740 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff86800cca0 tx=0x7ff86800e720 comp rx=0 tx=0).stop 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- 172.21.15.46:0/1758898267 shutdown_connections 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5cda80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- 172.21.15.46:0/1758898267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- 172.21.15.46:0/1758898267 >> 172.21.15.46:0/1758898267 conn(0x561c3e5baf60 msgr2=0x561c3e5bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- 172.21.15.46:0/1758898267 shutdown_connections 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- 172.21.15.46:0/1758898267 wait complete. 2024-09-12T02:50:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- start start 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 Processor -- start 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5c1850 con 0x561c3e5bf1d0 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c3e4dff20 con 0x561c3e5cdfc0 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5c19d0 con 0x561c3e5be8b0 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.991+0000 7ff87ed11740 10 client.0 _ll_register_callbacks cb 0x561c3e4a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40680/0 (socket says 172.21.15.46:40680) 2024-09-12T02:50:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 -- 172.21.15.46:0/501673520 learned_addr learned my addr 172.21.15.46:0/501673520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff875d9b640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff875d9b640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 -- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 msgr2=0x561c3e5c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 -- 172.21.15.46:0/501673520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 msgr2=0x561c3e5c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 -- 172.21.15.46:0/501673520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3e2962b0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff875d9b640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87659c640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c9e0 tx=0x7ff8680048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476c1610f48cf45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868030060 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7ff868004ed0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8680054d0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.992+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff868037050 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 10 client.15165 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561c3e5d8090 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8680054d0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff86802bae0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 10 client.15165 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 10 client.15165 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 20 client.15165 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e5da350 0x561c3e5db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 4 client.15165 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 --> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3e532600 con 0x561c3e5da350 2024-09-12T02:50:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.993+0000 7ff875d9b640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e5da350 0x561c3e5db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.994+0000 7ff875d9b640 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e5da350 0x561c3e5db7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fb74f9bd8bb2f76 server_cookie=5c24dbe27f2ddc16 in_seq=0 out_seq=0 2024-09-12T02:50:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.994+0000 7ff85dffb640 10 client.15165 ms_handle_connect on v2:172.21.15.196:6836/756659635 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.994+0000 7ff85dffb640 1 -- 172.21.15.46:0/501673520 <== mds.2 v2:172.21.15.196:6836/756659635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c3e532600 con 0x561c3e5da350 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.994+0000 7ff85dffb640 10 client.15165 handle_command_reply: tid=0 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 client.15165 shutdown 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 msgr2=0x561c3e5c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff86800c9e0 tx=0x7ff8680048b0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 shutdown_connections 2024-09-12T02:50:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e5da350 0x561c3e5db7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5cdfc0 0x561c3e5c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5bf1d0 0x561c3e5c7350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff85dffb640 0 client.15165 ms_handle_reset on v2:172.21.15.196:6836/756659635 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 --2- 172.21.15.46:0/501673520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5be8b0 0x561c3e5c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 >> 172.21.15.46:0/501673520 conn(0x561c3e5baf60 msgr2=0x561c3e5c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 shutdown_connections 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 1 -- 172.21.15.46:0/501673520 wait complete. 2024-09-12T02:50:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:00.995+0000 7ff87ed11740 20 client.15165 trim_cache size 0 max 16384 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 Processor -- start 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 -- start start 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5d9b00 con 0x561c3e5be8b0 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5da1d0 con 0x561c3e5bf1d0 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff87ed11740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5da8a0 con 0x561c3e5cdfc0 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40690/0 (socket says 172.21.15.46:40690) 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 -- 172.21.15.46:0/2014962079 learned_addr learned my addr 172.21.15.46:0/2014962079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 -- 172.21.15.46:0/2014962079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 msgr2=0x561c3e5e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 -- 172.21.15.46:0/2014962079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 msgr2=0x561c3e5e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 -- 172.21.15.46:0/2014962079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3e5baa00 con 0x561c3e5be8b0 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff875d9b640 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff860005fd0 tx=0x7ff860004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982b1dae8ca94792 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.013+0000 7ff85ffff640 1 -- 172.21.15.46:0/2014962079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86001c040 con 0x561c3e5be8b0 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff85ffff640 1 -- 172.21.15.46:0/2014962079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff860011d40 con 0x561c3e5be8b0 2024-09-12T02:50:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff85ffff640 1 -- 172.21.15.46:0/2014962079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860003a50 con 0x561c3e5be8b0 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- 172.21.15.46:0/2014962079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 msgr2=0x561c3e5c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff860005fd0 tx=0x7ff860004730 comp rx=0 tx=0).stop 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- 172.21.15.46:0/2014962079 shutdown_connections 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 --2- 172.21.15.46:0/2014962079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5c76e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- 172.21.15.46:0/2014962079 >> 172.21.15.46:0/2014962079 conn(0x561c3e5baf60 msgr2=0x561c3e5c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- 172.21.15.46:0/2014962079 shutdown_connections 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- 172.21.15.46:0/2014962079 wait complete. 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.014+0000 7ff87ed11740 1 -- start start 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 Processor -- start 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5db640 con 0x561c3e5be8b0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5daf70 con 0x561c3e5bf1d0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c3e5dbd10 con 0x561c3e5cdfc0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87ed11740 10 client.0 _ll_register_callbacks cb 0x7ff86000b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40692/0 (socket says 172.21.15.46:40692) 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 -- 172.21.15.46:0/3536375203 learned_addr learned my addr 172.21.15.46:0/3536375203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87659c640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 -- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 msgr2=0x561c3e5e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 -- 172.21.15.46:0/3536375203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 msgr2=0x561c3e5e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:01.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 -- 172.21.15.46:0/3536375203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3e5d3f20 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff87659c640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff875d9b640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b420 tx=0x7ff860011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4765ba69b1f6784 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.015+0000 7ff85dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860005020 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7ff8600051c0 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff860011d40 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff860037050 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 10 client.15171 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561c3e644c70 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff860011d40 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff86001f380 con 0x561c3e5be8b0 2024-09-12T02:50:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 10 client.15171 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 10 client.15171 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 20 client.15171 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e649470 0x561c3e648180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 4 client.15171 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.016+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 --> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] -- command(tid 0: {"prefix": "status"}) -- 0x561c3e52cb10 con 0x561c3e649470 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87659c640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e649470 0x561c3e648180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87659c640 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e649470 0x561c3e648180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2a96679346a3e04 server_cookie=1dbbe0903c393b6c in_seq=0 out_seq=0 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff85dffb640 10 client.15171 ms_handle_connect on v2:172.21.15.196:6836/756659635 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff85dffb640 1 -- 172.21.15.46:0/3536375203 <== mds.2 v2:172.21.15.196:6836/756659635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561c3e52cb10 con 0x561c3e649470 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff85dffb640 10 client.15171 handle_command_reply: tid=0 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 client.15171 shutdown 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 msgr2=0x561c3e5d6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b420 tx=0x7ff860011890 comp rx=0 tx=0).stop 2024-09-12T02:50:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 shutdown_connections 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:6836/756659635,v1:172.21.15.196:6837/756659635] conn(0x561c3e649470 0x561c3e648180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff85dffb640 0 client.15171 ms_handle_reset on v2:172.21.15.196:6836/756659635 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c3e5cdfc0 0x561c3e5e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561c3e5bf1d0 0x561c3e5e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 --2- 172.21.15.46:0/3536375203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c3e5be8b0 0x561c3e5d6400 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.017+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 >> 172.21.15.46:0/3536375203 conn(0x561c3e5baf60 msgr2=0x561c3e5ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 shutdown_connections 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87ed11740 1 -- 172.21.15.46:0/3536375203 wait complete. 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87ed11740 20 client.15171 trim_cache size 0 max 16384 2024-09-12T02:50:01.023 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24460, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 101.791970792, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 164.56945677900001 2024-09-12T02:50:01.024 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 msgr2=0x7ff8640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 0x7ff8640a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ee740 tx=0x7ff8700e7cc0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 msgr2=0x7ff87819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff870031040 tx=0x7ff870003ba0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 shutdown_connections 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff878106f10 0x7ff87819cbc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff8640a2b80 0x7ff8640a5040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff878102790 0x7ff878198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 --2- 172.21.15.46:0/1046970385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff878101de0 0x7ff8781982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 >> 172.21.15.46:0/1046970385 conn(0x7ff8780fd8e0 msgr2=0x7ff8780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 shutdown_connections 2024-09-12T02:50:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:01.018+0000 7ff87c938640 1 -- 172.21.15.46:0/1046970385 wait complete. 2024-09-12T02:50:01.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:50:02.006 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:02.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:02 smithi046 ceph-mon[30951]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:50:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:02 smithi196 ceph-mon[34587]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 -- 172.21.15.46:0/3789853922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 msgr2=0x7f6b801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 --2- 172.21.15.46:0/3789853922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801067e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b7402a410 comp rx=0 tx=0).stop 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 -- 172.21.15.46:0/3789853922 shutdown_connections 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 --2- 172.21.15.46:0/3789853922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80106f10 0x7f6b8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 --2- 172.21.15.46:0/3789853922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801067e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 --2- 172.21.15.46:0/3789853922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80101de0 0x7f6b801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 -- 172.21.15.46:0/3789853922 >> 172.21.15.46:0/3789853922 conn(0x7f6b800fd8c0 msgr2=0x7f6b800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 -- 172.21.15.46:0/3789853922 shutdown_connections 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.491+0000 7f6b87f92640 1 -- 172.21.15.46:0/3789853922 wait complete. 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 Processor -- start 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 -- start start 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80101de0 0x7f6b80198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 0x7f6b8019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80111e30 con 0x7f6b80102790 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80111cb0 con 0x7f6b80106f10 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b87f92640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80111fb0 con 0x7f6b80101de0 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40708/0 (socket says 172.21.15.46:40708) 2024-09-12T02:50:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 -- 172.21.15.46:0/1680403890 learned_addr learned my addr 172.21.15.46:0/1680403890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80101de0 0x7f6b80198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 -- 172.21.15.46:0/1680403890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80101de0 msgr2=0x7f6b80198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80101de0 0x7f6b80198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 -- 172.21.15.46:0/1680403890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 msgr2=0x7f6b8019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 0x7f6b8019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 0x7f6b8019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8552d640 1 -- 172.21.15.46:0/1680403890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8019d320 con 0x7f6b80102790 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.492+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 0x7f6b8019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b8552d640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005ab0 tx=0x7f6b74003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b7effd640 1 -- 172.21.15.46:0/1680403890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7403f070 con 0x7f6b80102790 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b7effd640 1 -- 172.21.15.46:0/1680403890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b74031d60 con 0x7f6b80102790 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8019d5b0 con 0x7f6b80102790 2024-09-12T02:50:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b7effd640 1 -- 172.21.15.46:0/1680403890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74035030 con 0x7f6b80102790 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.493+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b801a4ef0 con 0x7f6b80102790 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.495+0000 7f6b7effd640 1 -- 172.21.15.46:0/1680403890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b74042070 con 0x7f6b80102790 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.495+0000 7f6b8a36b740 1 Processor -- start 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 -- start start 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5587fe8fb090 con 0x5587fe9949d0 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5587fe8a2a70 con 0x5587fe9952f0 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587fe8f7050 con 0x5587fe99fbb0 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40720/0 (socket says 172.21.15.46:40720) 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 -- 172.21.15.46:0/1095371830 learned_addr learned my addr 172.21.15.46:0/1095371830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8552d640 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b7effd640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 0x7f6b680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 -- 172.21.15.46:0/1095371830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 msgr2=0x5587fe9a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 -- 172.21.15.46:0/1095371830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 msgr2=0x5587fe99f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b8652f640 1 -- 172.21.15.46:0/1095371830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fe8af130 con 0x5587fe9949d0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.496+0000 7f6b7effd640 1 -- 172.21.15.46:0/1680403890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f6b740ef920 con 0x7f6b80102790 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.497+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 0x7f6b680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.497+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009b10 tx=0x7f6b7800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d86f63a19037c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.497+0000 7f6b67fff640 1 -- 172.21.15.46:0/1095371830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7801e670 con 0x5587fe9949d0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.497+0000 7f6b67fff640 1 -- 172.21.15.46:0/1095371830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b78016070 con 0x5587fe9949d0 2024-09-12T02:50:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.497+0000 7f6b67fff640 1 -- 172.21.15.46:0/1095371830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7801db30 con 0x5587fe9949d0 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- 172.21.15.46:0/1095371830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 msgr2=0x5587fe994db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009b10 tx=0x7f6b7800e2f0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8652f640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 0x7f6b680a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800f010 tx=0x7f6b78020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- 172.21.15.46:0/1095371830 shutdown_connections 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/1095371830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe994db0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- 172.21.15.46:0/1095371830 >> 172.21.15.46:0/1095371830 conn(0x5587fe990f60 msgr2=0x5587fe991340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- 172.21.15.46:0/1095371830 shutdown_connections 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- 172.21.15.46:0/1095371830 wait complete. 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 -- start start 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.498+0000 7f6b8a36b740 1 Processor -- start 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe99ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9a2780 con 0x5587fe9949d0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5587fe8b5f20 con 0x5587fe9952f0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9a2900 con 0x5587fe99fbb0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8a36b740 10 client.0 _ll_register_callbacks cb 0x5587fe87f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40730/0 (socket says 172.21.15.46:40730) 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 -- 172.21.15.46:0/4240120439 learned_addr learned my addr 172.21.15.46:0/4240120439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8552d640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8652f640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe99ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 -- 172.21.15.46:0/4240120439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 msgr2=0x5587fe99ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe99ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 -- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 msgr2=0x5587fe99c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 -- 172.21.15.46:0/4240120439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587fe66c2b0 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b8652f640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe99ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6b70011b80 tx=0x7f6b7000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be2956341f1d9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7001e060 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.499+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f6b7000a750 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b78016070 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6b70025050 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 10 client.15183 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5587fe9ae100 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b78016070 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6b70018030 con 0x5587fe9949d0 2024-09-12T02:50:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 10 client.15183 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 10 client.15183 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 20 client.15183 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fe9b0770 0x5587fe9b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 4 client.15183 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 --> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587fe908600 con 0x5587fe9b0770 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.500+0000 7f6b8552d640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fe9b0770 0x5587fe9b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.501+0000 7f6b657fa640 10 client.15183 ms_handle_connect on v2:172.21.15.145:6838/1761070564 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.501+0000 7f6b8552d640 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fe9b0770 0x5587fe9b1be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b71fc90a3324741 server_cookie=c274adbceaa505f0 in_seq=0 out_seq=0 2024-09-12T02:50:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b657fa640 1 -- 172.21.15.46:0/4240120439 <== mds.1 v2:172.21.15.145:6838/1761070564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587fe908600 con 0x5587fe9b0770 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b657fa640 10 client.15183 handle_command_reply: tid=0 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 client.15183 shutdown 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 msgr2=0x5587fe99f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6b70011b80 tx=0x7f6b7000f790 comp rx=0 tx=0).stop 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 shutdown_connections 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fe9b0770 0x5587fe9b1be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe99ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe99c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b657fa640 0 client.15183 ms_handle_reset on v2:172.21.15.145:6838/1761070564 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/4240120439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe99f410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 >> 172.21.15.46:0/4240120439 conn(0x5587fe990f60 msgr2=0x5587fe996c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 shutdown_connections 2024-09-12T02:50:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 1 -- 172.21.15.46:0/4240120439 wait complete. 2024-09-12T02:50:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.502+0000 7f6b8a36b740 20 client.15183 trim_cache size 0 max 16384 2024-09-12T02:50:02.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 Processor -- start 2024-09-12T02:50:02.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 -- start start 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9ba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9aff20 con 0x5587fe9949d0 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9b05f0 con 0x5587fe9952f0 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9b0cc0 con 0x5587fe99fbb0 2024-09-12T02:50:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40746/0 (socket says 172.21.15.46:40746) 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.520+0000 7f6b85d2e640 1 -- 172.21.15.46:0/931834911 learned_addr learned my addr 172.21.15.46:0/931834911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b8552d640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b8652f640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 -- 172.21.15.46:0/931834911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 msgr2=0x5587fe9ba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9ba400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 -- 172.21.15.46:0/931834911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 msgr2=0x5587fe9b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 -- 172.21.15.46:0/931834911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fe99d2f0 con 0x5587fe9949d0 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b8552d640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b8652f640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9ba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c350 tx=0x7f6b70005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4112c5ed6c445 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b677fe640 1 -- 172.21.15.46:0/931834911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b700049e0 con 0x5587fe9949d0 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b677fe640 1 -- 172.21.15.46:0/931834911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7002e070 con 0x5587fe9949d0 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.521+0000 7f6b677fe640 1 -- 172.21.15.46:0/931834911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000a8d0 con 0x5587fe9949d0 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- 172.21.15.46:0/931834911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 msgr2=0x5587fe9b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c350 tx=0x7f6b70005e90 comp rx=0 tx=0).stop 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- 172.21.15.46:0/931834911 shutdown_connections 2024-09-12T02:50:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fe9ba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/931834911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b1be0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- 172.21.15.46:0/931834911 >> 172.21.15.46:0/931834911 conn(0x5587fe990f60 msgr2=0x5587fe99b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- 172.21.15.46:0/931834911 shutdown_connections 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- 172.21.15.46:0/931834911 wait complete. 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 -- start start 2024-09-12T02:50:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.522+0000 7f6b8a36b740 1 Processor -- start 2024-09-12T02:50:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fea18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9b1a60 con 0x5587fe9949d0 2024-09-12T02:50:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9b1390 con 0x5587fe9952f0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587fe9b2130 con 0x5587fe99fbb0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8a36b740 10 client.0 _ll_register_callbacks cb 0x7f6b700079f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40758/0 (socket says 172.21.15.46:40758) 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 -- 172.21.15.46:0/817565282 learned_addr learned my addr 172.21.15.46:0/817565282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8552d640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b8652f640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fea18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 -- 172.21.15.46:0/817565282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 msgr2=0x5587fea18e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fea18e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 -- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 msgr2=0x5587fe9b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 -- 172.21.15.46:0/817565282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587fea193b0 con 0x5587fe9949d0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b85d2e640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6b700076d0 tx=0x7f6b7000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22081d1ffd407175 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.523+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7001e040 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f6b7000d340 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7002e070 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6b70044050 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 10 client.15189 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5587fea1a480 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7002e070 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6b70018040 con 0x5587fe9949d0 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 10 client.15189 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 10 client.15189 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 20 client.15189 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fea1edc0 0x5587fea1dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 4 client.15189 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.524+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 --> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] -- command(tid 0: {"prefix": "status"}) -- 0x5587fe902b10 con 0x5587fea1edc0 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8552d640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fea1edc0 0x5587fea1dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8552d640 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fea1edc0 0x5587fea1dad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=363dd55dd20f821c server_cookie=d1e656b80af8a0fb in_seq=0 out_seq=0 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b657fa640 10 client.15189 ms_handle_connect on v2:172.21.15.145:6838/1761070564 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b657fa640 1 -- 172.21.15.46:0/817565282 <== mds.1 v2:172.21.15.145:6838/1761070564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5587fe902b10 con 0x5587fea1edc0 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b657fa640 10 client.15189 handle_command_reply: tid=0 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8a36b740 1 client.15189 shutdown 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 msgr2=0x5587fe9b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6b700076d0 tx=0x7f6b7000ded0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.525+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 shutdown_connections 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:6838/1761070564,v1:172.21.15.145:6839/1761070564] conn(0x5587fea1edc0 0x5587fea1dad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587fe99fbb0 0x5587fea18e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5587fe9952f0 0x5587fe9b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b657fa640 0 client.15189 ms_handle_reset on v2:172.21.15.145:6838/1761070564 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 --2- 172.21.15.46:0/817565282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5587fe9949d0 0x5587fe9b4fa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 >> 172.21.15.46:0/817565282 conn(0x5587fe990f60 msgr2=0x5587fe99b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 shutdown_connections 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 1 -- 172.21.15.46:0/817565282 wait complete. 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b8a36b740 20 client.15189 trim_cache size 0 max 16384 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24476, 2024-09-12T02:50:02.534 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:50:02.535 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:50:02.535 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:50:02.536 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 104.312200738, 2024-09-12T02:50:02.536 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:50:02.536 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:50:02.536 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:50:02.536 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 151.488332187 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 msgr2=0x7f6b680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 0x7f6b680a50c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800f010 tx=0x7f6b78020000 comp rx=0 tx=0).stop 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 msgr2=0x7f6b801988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005ab0 tx=0x7f6b74003cd0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 shutdown_connections 2024-09-12T02:50:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b80106f10 0x7f6b8019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b80102790 0x7f6b801988b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f6b680a2c00 0x7f6b680a50c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 --2- 172.21.15.46:0/1680403890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b80101de0 0x7f6b80198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 >> 172.21.15.46:0/1680403890 conn(0x7f6b800fd8c0 msgr2=0x7f6b800fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 shutdown_connections 2024-09-12T02:50:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:02.526+0000 7f6b87f92640 1 -- 172.21.15.46:0/1680403890 wait complete. 2024-09-12T02:50:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:02 smithi145 ceph-mon[34674]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:50:03.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:50:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 -- 172.21.15.46:0/2626682548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640100a80 msgr2=0x7f5640100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 --2- 172.21.15.46:0/2626682548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640100a80 0x7f5640100ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5630009930 tx=0x7f563002a420 comp rx=0 tx=0).stop 2024-09-12T02:50:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 -- 172.21.15.46:0/2626682548 shutdown_connections 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 --2- 172.21.15.46:0/2626682548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 0x7f564010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 --2- 172.21.15.46:0/2626682548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640100a80 0x7f5640100ee0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 --2- 172.21.15.46:0/2626682548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640106a80 0x7f5640106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 -- 172.21.15.46:0/2626682548 >> 172.21.15.46:0/2626682548 conn(0x7f56400fc7a0 msgr2=0x7f56400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 -- 172.21.15.46:0/2626682548 shutdown_connections 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.035+0000 7f56484f0640 1 -- 172.21.15.46:0/2626682548 wait complete. 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 Processor -- start 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 -- start start 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640100a80 0x7f56401983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 0x7f5640198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5640110d30 con 0x7f5640106a80 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5640110bb0 con 0x7f5640100a80 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f56484f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5640110eb0 con 0x7f5640101420 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f5646a8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f5646a8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40766/0 (socket says 172.21.15.46:40766) 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.036+0000 7f5646a8d640 1 -- 172.21.15.46:0/922467946 learned_addr learned my addr 172.21.15.46:0/922467946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5645a8b640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 0x7f5640198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f564628c640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640100a80 0x7f56401983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 -- 172.21.15.46:0/922467946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 msgr2=0x7f5640198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 0x7f5640198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 -- 172.21.15.46:0/922467946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640100a80 msgr2=0x7f56401983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640100a80 0x7f56401983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 -- 172.21.15.46:0/922467946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564019d370 con 0x7f5640106a80 2024-09-12T02:50:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f5646a8d640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f563800eb10 tx=0x7f563800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f56377fe640 1 -- 172.21.15.46:0/922467946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638004b20 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564019d660 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f56377fe640 1 -- 172.21.15.46:0/922467946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5638004cc0 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f56377fe640 1 -- 172.21.15.46:0/922467946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638010690 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.037+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56401a4f40 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.039+0000 7f56377fe640 1 -- 172.21.15.46:0/922467946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5638005020 con 0x7f5640106a80 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 Processor -- start 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 -- start start 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb099d0 0x55e4dcb09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4dca70090 con 0x55e4dcb14bb0 2024-09-12T02:50:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4dca17a70 con 0x55e4dcb0a2f0 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564a8c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4dca6c050 con 0x55e4dcb099d0 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40772/0 (socket says 172.21.15.46:40772) 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f5645a8b640 1 -- 172.21.15.46:0/168626832 learned_addr learned my addr 172.21.15.46:0/168626832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f5645a8b640 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb099d0 0x55e4dcb09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.040+0000 7f564628c640 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f56377fe640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 0x7f562c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 -- 172.21.15.46:0/168626832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb099d0 msgr2=0x55e4dcb09db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb099d0 0x55e4dcb09db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 -- 172.21.15.46:0/168626832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 msgr2=0x55e4dcb14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 -- 172.21.15.46:0/168626832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4dca24130 con 0x55e4dcb14bb0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5646a8d640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 0x7f562c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5645a8b640 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5630009930 tx=0x7f5630005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a7bef21137034 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f56377fe640 1 -- 172.21.15.46:0/922467946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f56380cfac0 con 0x7f5640106a80 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5634ff9640 1 -- 172.21.15.46:0/168626832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630035070 con 0x55e4dcb14bb0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5634ff9640 1 -- 172.21.15.46:0/168626832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5630004030 con 0x55e4dcb14bb0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.041+0000 7f5634ff9640 1 -- 172.21.15.46:0/168626832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630005280 con 0x55e4dcb14bb0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- 172.21.15.46:0/168626832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 msgr2=0x55e4dcb16fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f5646a8d640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 0x7f562c0a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f56380049d0 tx=0x7f56380c6ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5630009930 tx=0x7f5630005fd0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- 172.21.15.46:0/168626832 shutdown_connections 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb16fa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 --2- 172.21.15.46:0/168626832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb099d0 0x55e4dcb09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- 172.21.15.46:0/168626832 >> 172.21.15.46:0/168626832 conn(0x55e4dcb05f60 msgr2=0x55e4dcb06340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- 172.21.15.46:0/168626832 shutdown_connections 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- 172.21.15.46:0/168626832 wait complete. 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 -- start start 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.042+0000 7f564a8c9740 1 Processor -- start 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb11d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb17720 con 0x55e4dcb099d0 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4dca2af20 con 0x55e4dcb14bb0 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb178a0 con 0x55e4dcb0a2f0 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564a8c9740 10 client.0 _ll_register_callbacks cb 0x55e4dc9f4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:04.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40784/0 (socket says 172.21.15.46:40784) 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 -- 172.21.15.46:0/2031650816 learned_addr learned my addr 172.21.15.46:0/2031650816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb11d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f564628c640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 -- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb0a2f0 msgr2=0x55e4dcb117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 -- 172.21.15.46:0/2031650816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb14bb0 msgr2=0x55e4dcb11d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb11d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 -- 172.21.15.46:0/2031650816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4dc7e12b0 con 0x55e4dcb099d0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5645a8b640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f563002a460 tx=0x7f56300098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8872eadd3f695263 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.043+0000 7f5629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630035070 con 0x55e4dcb099d0 2024-09-12T02:50:04.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5630004030 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f5630031cf0 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f56300044b0 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 10 client.15201 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4dcb230c0 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5630004030 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5630004c30 con 0x55e4dcb099d0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 10 client.15201 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 20 client.15201 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb24410 0x55e4dcb26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 4 client.15201 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.044+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4dca7d600 con 0x55e4dcb24410 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.045+0000 7f564628c640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb24410 0x55e4dcb26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.045+0000 7f564628c640 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb24410 0x55e4dcb26550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ce23fb880377aa server_cookie=88eed873cdd003e3 in_seq=0 out_seq=0 2024-09-12T02:50:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.045+0000 7f5629ffb640 10 client.15201 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f5629ffb640 1 -- 172.21.15.46:0/2031650816 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4dca7d600 con 0x55e4dcb24410 2024-09-12T02:50:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f5629ffb640 10 client.15201 handle_command_reply: tid=0 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 client.15201 shutdown 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 msgr2=0x55e4dcb144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f563002a460 tx=0x7f56300098a0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 shutdown_connections 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb24410 0x55e4dcb26550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb11d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 --2- 172.21.15.46:0/2031650816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb144a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 >> 172.21.15.46:0/2031650816 conn(0x55e4dcb05f60 msgr2=0x55e4dcb0bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f5629ffb640 0 client.15201 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 shutdown_connections 2024-09-12T02:50:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 1 -- 172.21.15.46:0/2031650816 wait complete. 2024-09-12T02:50:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.046+0000 7f564a8c9740 20 client.15201 trim_cache size 0 max 16384 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 Processor -- start 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 -- start start 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb24890 con 0x55e4dcb099d0 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb24f60 con 0x55e4dcb0a2f0 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.064+0000 7f564a8c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb25630 con 0x55e4dcb14bb0 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40798/0 (socket says 172.21.15.46:40798) 2024-09-12T02:50:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 -- 172.21.15.46:0/3936353981 learned_addr learned my addr 172.21.15.46:0/3936353981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 -- 172.21.15.46:0/3936353981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 msgr2=0x55e4dcb108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 -- 172.21.15.46:0/3936353981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 msgr2=0x55e4dcb110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 -- 172.21.15.46:0/3936353981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4dcb14810 con 0x55e4dcb099d0 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f563002abd0 tx=0x7f56300037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ee1e5e72608ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f562bfff640 1 -- 172.21.15.46:0/3936353981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630035020 con 0x55e4dcb099d0 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f562bfff640 1 -- 172.21.15.46:0/3936353981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5630031e30 con 0x55e4dcb099d0 2024-09-12T02:50:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.065+0000 7f562bfff640 1 -- 172.21.15.46:0/3936353981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630032600 con 0x55e4dcb099d0 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- 172.21.15.46:0/3936353981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 msgr2=0x55e4dcb26550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f563002abd0 tx=0x7f56300037e0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- 172.21.15.46:0/3936353981 shutdown_connections 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3936353981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb26550 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- 172.21.15.46:0/3936353981 >> 172.21.15.46:0/3936353981 conn(0x55e4dcb05f60 msgr2=0x55e4dcb06340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- 172.21.15.46:0/3936353981 shutdown_connections 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- 172.21.15.46:0/3936353981 wait complete. 2024-09-12T02:50:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 -- start start 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.066+0000 7f564a8c9740 1 Processor -- start 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb27e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb263d0 con 0x55e4dcb099d0 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb25d00 con 0x55e4dcb0a2f0 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4dcb26aa0 con 0x55e4dcb14bb0 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564a8c9740 10 client.0 _ll_register_callbacks cb 0x7f563c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40808/0 (socket says 172.21.15.46:40808) 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 -- 172.21.15.46:0/3239977716 learned_addr learned my addr 172.21.15.46:0/3239977716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564628c640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 -- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 msgr2=0x55e4dcb27e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 -- 172.21.15.46:0/3239977716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 msgr2=0x55e4dcb277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 -- 172.21.15.46:0/3239977716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4dcb285f0 con 0x55e4dcb099d0 2024-09-12T02:50:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f564628c640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5645a8b640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56300322e0 tx=0x7f56300098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d24b45b3510df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.067+0000 7f5629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563006c060 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f5630003e60 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5630031e30 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5630035070 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 10 client.15207 fetch_fsmap learned FSMap version 24 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4dcb97120 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5630031e30 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5630005090 con 0x55e4dcb099d0 2024-09-12T02:50:04.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 10 client.15207 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 20 client.15207 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb9b7b0 0x55e4dcb9a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 4 client.15207 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.068+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "status"}) -- 0x55e4dca77b10 con 0x55e4dcb9b7b0 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564628c640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb9b7b0 0x55e4dcb9a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f5629ffb640 10 client.15207 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:04.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564628c640 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb9b7b0 0x55e4dcb9a4c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ea5ae65643595 server_cookie=d9c21026405fce7c in_seq=0 out_seq=0 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f5629ffb640 1 -- 172.21.15.46:0/3239977716 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e4dca77b10 con 0x55e4dcb9b7b0 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f5629ffb640 10 client.15207 handle_command_reply: tid=0 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 client.15207 shutdown 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 msgr2=0x55e4dcb2f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56300322e0 tx=0x7f56300098e0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 shutdown_connections 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e4dcb9b7b0 0x55e4dcb9a4c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4dcb14bb0 0x55e4dcb27e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4dcb0a2f0 0x55e4dcb277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 --2- 172.21.15.46:0/3239977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4dcb099d0 0x55e4dcb2f050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 >> 172.21.15.46:0/3239977716 conn(0x55e4dcb05f60 msgr2=0x55e4dcb108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.069+0000 7f5629ffb640 0 client.15207 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 shutdown_connections 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f564a8c9740 1 -- 172.21.15.46:0/3239977716 wait complete. 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f564a8c9740 20 client.15207 trim_cache size 0 max 16384 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "2691fc0c-70b0-11ef-bcea-c7b262605968", 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24478, 2024-09-12T02:50:04.078 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 141.967994385, 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 162.21549776500001 2024-09-12T02:50:04.079 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 msgr2=0x7f562c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 0x7f562c0a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f56380049d0 tx=0x7f56380c6ad0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 msgr2=0x7f564019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f563800eb10 tx=0x7f563800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 shutdown_connections 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5640106a80 0x7f564019cc90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f562c0a2b80 0x7f562c0a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5640101420 0x7f5640198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 --2- 172.21.15.46:0/922467946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5640100a80 0x7f56401983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 >> 172.21.15.46:0/922467946 conn(0x7f56400fc7a0 msgr2=0x7f564006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.070+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 shutdown_connections 2024-09-12T02:50:04.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.071+0000 7f56484f0640 1 -- 172.21.15.46:0/922467946 wait complete. 2024-09-12T02:50:04.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:04 smithi046 ceph-mon[30951]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:50:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:04 smithi196 ceph-mon[34587]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:50:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:04 smithi145 ceph-mon[34674]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:50:04.793 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T02:50:04.804 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T02:50:04.805 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T02:50:04.805 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T02:50:04.805 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T02:50:04.806 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:50:04.806 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:50:04.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T02:50:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.917+0000 7ff210d23640 1 -- 172.21.15.46:0/3393459975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c104f10 msgr2=0x7ff20c107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.917+0000 7ff210d23640 1 --2- 172.21.15.46:0/3393459975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c104f10 0x7ff20c107300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff200009900 tx=0x7ff20002a410 comp rx=0 tx=0).stop 2024-09-12T02:50:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 -- 172.21.15.46:0/3393459975 shutdown_connections 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 --2- 172.21.15.46:0/3393459975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c107840 0x7ff20c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 --2- 172.21.15.46:0/3393459975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c104f10 0x7ff20c107300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 --2- 172.21.15.46:0/3393459975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 0x7ff20c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 -- 172.21.15.46:0/3393459975 >> 172.21.15.46:0/3393459975 conn(0x7ff20c0fc1e0 msgr2=0x7ff20c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 -- 172.21.15.46:0/3393459975 shutdown_connections 2024-09-12T02:50:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 -- 172.21.15.46:0/3393459975 wait complete. 2024-09-12T02:50:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 Processor -- start 2024-09-12T02:50:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.918+0000 7ff210d23640 1 -- start start 2024-09-12T02:50:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 0x7ff20c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 0x7ff20c100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d6a0 con 0x7ff20c107840 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d520 con 0x7ff20c1025e0 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff210d23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d820 con 0x7ff20c104f10 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52866/0 (socket says 172.21.15.46:52866) 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 -- 172.21.15.46:0/1023463109 learned_addr learned my addr 172.21.15.46:0/1023463109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20a59c640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 0x7ff20c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff209d9b640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 0x7ff20c100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 -- 172.21.15.46:0/1023463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 msgr2=0x7ff20c100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 0x7ff20c100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 -- 172.21.15.46:0/1023463109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 msgr2=0x7ff20c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 0x7ff20c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 -- 172.21.15.46:0/1023463109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c101360 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff209d9b640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 0x7ff20c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.919+0000 7ff20ad9d640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9d0 tx=0x7ff20400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.920+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204004d20 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.920+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff204005020 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.920+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c1a1e20 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.920+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040116b0 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.920+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c1a2230 con 0x7ff20c107840 2024-09-12T02:50:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.921+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff20c0fdcd0 con 0x7ff20c107840 2024-09-12T02:50:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.922+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2040051d0 con 0x7ff20c107840 2024-09-12T02:50:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.922+0000 7ff1fb7fe640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.923+0000 7ff20a59c640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.923+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7ff2040cf9d0 con 0x7ff20c107840 2024-09-12T02:50:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.923+0000 7ff20a59c640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00abe0 tx=0x7ff1fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:04.926+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff204016030 con 0x7ff20c107840 2024-09-12T02:50:05.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.155+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ff20c062e90 con 0x7ff20c107840 2024-09-12T02:50:05.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.182+0000 7ff1fb7fe640 1 -- 172.21.15.46:0/1023463109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7ff20409c550 con 0x7ff20c107840 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 msgr2=0x7ff1f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00abe0 tx=0x7ff1fc00a2d0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 msgr2=0x7ff20c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9d0 tx=0x7ff20400edd0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 shutdown_connections 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20c107840 0x7ff20c100ad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff20c104f10 0x7ff20c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 --2- 172.21.15.46:0/1023463109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff20c1025e0 0x7ff20c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 >> 172.21.15.46:0/1023463109 conn(0x7ff20c0fc1e0 msgr2=0x7ff20c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.184+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 shutdown_connections 2024-09-12T02:50:05.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:05.185+0000 7ff210d23640 1 -- 172.21.15.46:0/1023463109 wait complete. 2024-09-12T02:50:05.201 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-12T02:50:05.213 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-12T02:50:05.214 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-12T02:50:05.214 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-12T02:50:05.214 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T02:50:05.214 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:05.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:05.214 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-12T02:50:05.231 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T02:50:05.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:05.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T02:50:05.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:05.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-12T02:50:05.411 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-12T02:50:05.411 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T02:50:05.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T02:50:05.438 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T02:50:05.439 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setuser ceph since I am not root 2024-09-12T02:50:05.439 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ignoring --setgroup ceph since I am not root 2024-09-12T02:50:05.439 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.343+0000 7fe2abb66040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:50:05.439 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.407+0000 7fe2abb66040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: ignoring --setuser ceph since I am not root 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: ignoring --setgroup ceph since I am not root 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-mgr[36396]: -- 172.21.15.196:0/3565706291 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589557ad0e0 con 0x55895578b400 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.335+0000 7f206f067040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:50:05.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.392+0000 7f206f067040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: ignoring --setuser ceph since I am not root 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: ignoring --setgroup ceph since I am not root 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.336+0000 7f3e905b3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.393+0000 7f3e905b3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:05.543 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.541+0000 7f3e905b3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:50:05.681 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:05.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.569+0000 7fe2abb66040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:50:05.783 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.645+0000 7fe2abb66040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:50:05.860 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.608+0000 7f3e905b3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:50:05.860 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.754+0000 7f3e905b3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:05.892 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.540+0000 7f206f067040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:50:05.892 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.606+0000 7f206f067040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:50:05.892 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.752+0000 7f206f067040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:05.892 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:05.890+0000 7f206f067040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:50:06.104 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.806+0000 7fe2abb66040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:06.105 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:05 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:05.956+0000 7fe2abb66040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:50:06.183 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:05.894+0000 7f3e905b3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:50:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.189+0000 7f3730ff6640 1 -- 172.21.15.46:0/2512222414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c1036e0 msgr2=0x7f372c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.189+0000 7f3730ff6640 1 --2- 172.21.15.46:0/2512222414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c1036e0 0x7f372c107b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3720009930 tx=0x7f372002a420 comp rx=0 tx=0).stop 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 -- 172.21.15.46:0/2512222414 shutdown_connections 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 --2- 172.21.15.46:0/2512222414 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1082a0 0x7f372c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 --2- 172.21.15.46:0/2512222414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c1036e0 0x7f372c107b90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 --2- 172.21.15.46:0/2512222414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c102d30 0x7f372c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 -- 172.21.15.46:0/2512222414 >> 172.21.15.46:0/2512222414 conn(0x7f372c0fc7d0 msgr2=0x7f372c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 -- 172.21.15.46:0/2512222414 shutdown_connections 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 -- 172.21.15.46:0/2512222414 wait complete. 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 Processor -- start 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.190+0000 7f3730ff6640 1 -- start start 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 0x7f372c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c1082a0 0x7f372c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f372c111f50 con 0x7f372c102d30 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f372c111dd0 con 0x7f372c1036e0 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3730ff6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1120d0 con 0x7f372c1082a0 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52906/0 (socket says 172.21.15.46:52906) 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 -- 172.21.15.46:0/621134281 learned_addr learned my addr 172.21.15.46:0/621134281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3729d9b640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 0x7f372c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372ad9d640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c1082a0 0x7f372c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c1082a0 msgr2=0x7f372c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c1082a0 0x7f372c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 msgr2=0x7f372c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 0x7f372c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 -- 172.21.15.46:0/621134281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c19d4a0 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f3729d9b640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 0x7f372c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.191+0000 7f372a59c640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.192+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00cd80 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.192+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371c0054d0 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.192+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c19d790 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.192+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c010490 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.192+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a5070 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.193+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c103110 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.194+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f371c005020 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.195+0000 7f371b7fe640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 0x7f37140a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.195+0000 7f3729d9b640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 msgr2=0x7f37140a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.195+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f371c0cf810 con 0x7f372c102d30 2024-09-12T02:50:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.195+0000 7f3729d9b640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 0x7f37140a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.199+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f371c09c390 con 0x7f372c102d30 2024-09-12T02:50:06.358 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:06 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T02:50:06.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:06 smithi046 ceph-mon[30951]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T02:50:06.359 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:06.294+0000 7fe2abb66040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:50:06.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.396+0000 7f3729d9b640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 msgr2=0x7f37140a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:06.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.396+0000 7f3729d9b640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 0x7f37140a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:50:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.413+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f372c10a2b0 con 0x7f372c102d30 2024-09-12T02:50:06.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.414+0000 7f371b7fe640 1 -- 172.21.15.46:0/621134281 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f371c09bc90 con 0x7f372c102d30 2024-09-12T02:50:06.414 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T02:50:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 msgr2=0x7f37140a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 0x7f37140a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 msgr2=0x7f372c1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00edf0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 shutdown_connections 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f37140a2bd0 0x7f37140a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f372c1082a0 0x7f372c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372c1036e0 0x7f372c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 --2- 172.21.15.46:0/621134281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c102d30 0x7f372c1984f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 >> 172.21.15.46:0/621134281 conn(0x7f372c0fc7d0 msgr2=0x7f372c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 shutdown_connections 2024-09-12T02:50:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:06.417+0000 7f3730ff6640 1 -- 172.21.15.46:0/621134281 wait complete. 2024-09-12T02:50:06.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:06 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T02:50:06.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:06 smithi196 ceph-mon[34587]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T02:50:06.448 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:06.202+0000 7f206f067040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:50:06.448 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:06.262+0000 7f206f067040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:50:06.454 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:06 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1023463109' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T02:50:06.454 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:06 smithi145 ceph-mon[34674]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T02:50:06.454 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:06 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:06.207+0000 7f3e905b3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:50:06.454 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:06 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:06.268+0000 7f3e905b3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:50:06.454 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:06 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:06.452+0000 7f3e905b3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:50:06.720 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:06.446+0000 7f206f067040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:50:06.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:06.358+0000 7fe2abb66040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:50:06.783 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:06.571+0000 7fe2abb66040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:50:07.148 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T02:50:07.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T02:50:07.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-12T02:50:07.148 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T02:50:07.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:07.149 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T02:50:07.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T02:50:07.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T02:50:07.149 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:50:07.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:07.149 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-12T02:50:07.206 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42153sec preferred_lft 42153sec 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:50:07.207 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-12T02:50:07.208 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T02:50:07.208 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:50:07.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:50:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:07.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:07.354 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T02:50:07.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:07 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/621134281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:50:07.380 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-12T02:50:07.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:07.381 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-12T02:50:07.439 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-12T02:50:07.440 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T02:50:07.440 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T02:50:07.440 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T02:50:07.440 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T02:50:07.440 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:50:07.441 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:50:07.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T02:50:07.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T02:50:07.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T02:50:07.441 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:50:07.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:07 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/621134281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:50:07.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:07 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:07.324+0000 7f206f067040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:50:07.471 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:07 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:07.390+0000 7f206f067040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:50:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:50:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:07.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:07.573 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-12T02:50:07.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:07.592 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-12T02:50:07.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:07 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/621134281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:50:07.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:07 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:07.335+0000 7f3e905b3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:50:07.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:07 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:07.403+0000 7f3e905b3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:50:07.650 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:50:07.650 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:50:07.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:50:07.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T02:50:07.651 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:50:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:50:07.749 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:07.520+0000 7fe2abb66040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:50:07.749 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:07 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:07.595+0000 7fe2abb66040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:50:07.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:07.772 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T02:50:07.772 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:50:07.772 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:50:07.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T02:50:07.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T02:50:07.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T02:50:07.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T02:50:07.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T02:50:07.773 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:50:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:50:07.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:07.899 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:50:07.899 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-12T02:50:07.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-12T02:50:07.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T02:50:07.900 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-12T02:50:07.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:50:07.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:07.980 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-12T02:50:07.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:50:07.980 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:07.996 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:50:07.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:50:07.997 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:08.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-12T02:50:08.123 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T02:50:08.180 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-12T02:50:08.180 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-12T02:50:08.181 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f0c19eb0b94f9aac184e195c7f6002cc1352e1ada35b3e3c6c03ec4e6f9f0589/merged 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ad9fc8f688582ce4273a5548a52775c61cc23484a3d48f6497ef5c22e5130af/merged 2024-09-12T02:50:08.182 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a27b6e2ef876def5568a0ef69271e77dbcc03bcf4664f6b949ed87a40de6b46b/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3a3cbbcb971750e1719fbacacda453e4a057ff24352230a8a3fd641caac9fe78/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/94a52bf189183ebdc3c84bdebaf8cb491f16f71cf57f6e443a7f1ac905c9458a/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ad996249b6177c62fc252717bde5789ead510efb33e13cac4ff07d9b8965a2ab/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/848ab539dead75324717f5809fdfd21e3d2c81405a32475f482fad361b374365/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/c00ac6eb3dc218102de5fbe213c6c2189702197b5132884c76a2d631dbb39741/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6e88b2d27417c36fe0f7cb64ed160327e936c22e5467e33c02b3d99461af6165/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a43b08778498f2caa9c2fc637ca3a82fbea959c9c67e179e391c322e38b77a1d/merged 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:50:08.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:08.184 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-12T02:50:08.236 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T02:50:08.237 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/7124dae5-f95d-40d9-b3df-bb43c74cb3cf --client_fs=cephfs) 2024-09-12T02:50:08.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-12T02:50:08.306 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T02:50:08.358 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.099+0000 7f206f067040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:50:08.359 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.215+0000 7f206f067040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-12T02:50:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T02:50:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f0c19eb0b94f9aac184e195c7f6002cc1352e1ada35b3e3c6c03ec4e6f9f0589/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ad9fc8f688582ce4273a5548a52775c61cc23484a3d48f6497ef5c22e5130af/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a27b6e2ef876def5568a0ef69271e77dbcc03bcf4664f6b949ed87a40de6b46b/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3a3cbbcb971750e1719fbacacda453e4a057ff24352230a8a3fd641caac9fe78/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/94a52bf189183ebdc3c84bdebaf8cb491f16f71cf57f6e443a7f1ac905c9458a/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ad996249b6177c62fc252717bde5789ead510efb33e13cac4ff07d9b8965a2ab/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/848ab539dead75324717f5809fdfd21e3d2c81405a32475f482fad361b374365/merged 2024-09-12T02:50:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/c00ac6eb3dc218102de5fbe213c6c2189702197b5132884c76a2d631dbb39741/merged 2024-09-12T02:50:08.366 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6e88b2d27417c36fe0f7cb64ed160327e936c22e5467e33c02b3d99461af6165/merged 2024-09-12T02:50:08.366 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a43b08778498f2caa9c2fc637ca3a82fbea959c9c67e179e391c322e38b77a1d/merged 2024-09-12T02:50:08.366 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-12T02:50:08.366 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:50:08.366 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:50:08.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:08.367 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-12T02:50:08.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-09-12T02:50:08.366+0000 7f66a980a480 -1 init, newargv = 0x55e96f1b4b80 newargc=15 2024-09-12T02:50:08.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[85429]: starting ceph client 2024-09-12T02:50:08.372 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.111+0000 7f3e905b3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:50:08.372 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.227+0000 7f3e905b3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:50:08.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[85429]: starting fuse 2024-09-12T02:50:08.396 INFO:teuthology.orchestra.run.smithi046.stdout:120 2024-09-12T02:50:08.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-12T02:50:08.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-09-12T02:50:08.396 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:50:08.397 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-09-12T02:50:08.398 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-09-12T02:50:08.518 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.85429.asok 2024-09-12T02:50:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:50:08.530 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85429.asok 2024-09-12T02:50:08.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:08.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85429.asok status 2024-09-12T02:50:08.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.357+0000 7f206f067040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:08.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.420+0000 7f206f067040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:50:08.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.567+0000 7f206f067040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:50:08.636 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.635+0000 7f206f067040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:50:08.650 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.371+0000 7f3e905b3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:08.650 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.433+0000 7f3e905b3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:50:08.650 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.581+0000 7f3e905b3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:50:08.651 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.649+0000 7f3e905b3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:50:08.657 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.376+0000 7fe2abb66040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:50:08.657 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.503+0000 7fe2abb66040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T02:50:08.668 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "85429", 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/volumes/qa/sv_0/7124dae5-f95d-40d9-b3df-bb43c74cb3cf" 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 15240, 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 15240 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-12T02:50:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2903137600 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2903137600 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.15240 192.168.0.1:0/2903137600", 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2903137600", 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 5, 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 24, 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 92, 2024-09-12T02:50:08.670 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-09-12T02:50:08.671 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-09-12T02:50:08.671 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-09-12T02:50:08.671 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T02:50:08.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:08.677 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-09-12T02:50:08.677 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-09-12T02:50:08.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:08.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:50:08.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:08.764 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T02:50:08.777 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:50:08.777 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:50:08.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T02:50:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 -- 172.21.15.46:0/3501777330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 msgr2=0x7f94801020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 --2- 172.21.15.46:0/3501777330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f94801020b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9470009900 tx=0x7f947002a410 comp rx=0 tx=0).stop 2024-09-12T02:50:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 -- 172.21.15.46:0/3501777330 shutdown_connections 2024-09-12T02:50:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 --2- 172.21.15.46:0/3501777330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94801025f0 0x7f9480108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 --2- 172.21.15.46:0/3501777330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f94801020b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 --2- 172.21.15.46:0/3501777330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9480100a40 0x7f9480100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 -- 172.21.15.46:0/3501777330 >> 172.21.15.46:0/3501777330 conn(0x7f94800fc1e0 msgr2=0x7f94800fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 -- 172.21.15.46:0/3501777330 shutdown_connections 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.910+0000 7f948615b640 1 -- 172.21.15.46:0/3501777330 wait complete. 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 Processor -- start 2024-09-12T02:50:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 -- start start 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9480100a40 0x7f9480199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94801025f0 0x7f94801a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f948010c570 con 0x7f9480101c30 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f948010c3f0 con 0x7f94801025f0 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f948615b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f948010c6f0 con 0x7f9480100a40 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f947effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f947effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52918/0 (socket says 172.21.15.46:52918) 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f947effd640 1 -- 172.21.15.46:0/764146336 learned_addr learned my addr 172.21.15.46:0/764146336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f947f7fe640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9480100a40 0x7f9480199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.911+0000 7f947ffff640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94801025f0 0x7f94801a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9480100a40 msgr2=0x7f9480199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9480100a40 0x7f9480199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94801025f0 msgr2=0x7f94801a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94801025f0 0x7f94801a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 -- 172.21.15.46:0/764146336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801a1b20 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947effd640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9470005940 tx=0x7f94700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947003f030 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94700318f0 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470035070 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801a1db0 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.912+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a2240 con 0x7f9480101c30 2024-09-12T02:50:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.913+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94801a2470 con 0x7f9480101c30 2024-09-12T02:50:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.915+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f9470042030 con 0x7f9480101c30 2024-09-12T02:50:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.915+0000 7f947cff9640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 0x7f946c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.915+0000 7f947f7fe640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 msgr2=0x7f946c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.915+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f94700efea0 con 0x7f9480101c30 2024-09-12T02:50:08.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.915+0000 7f947f7fe640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 0x7f946c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:08.918+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94700bc430 con 0x7f9480101c30 2024-09-12T02:50:08.919 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.694+0000 7f206f067040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:50:08.920 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.854+0000 7f206f067040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:50:08.920 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.919+0000 7f206f067040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:50:08.934 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.708+0000 7f3e905b3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:50:08.934 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.867+0000 7f3e905b3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:50:08.958 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.656+0000 7fe2abb66040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:08.958 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.722+0000 7fe2abb66040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:50:08.958 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.883+0000 7fe2abb66040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:50:08.958 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:08 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:08.957+0000 7fe2abb66040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:50:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.116+0000 7f947f7fe640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 msgr2=0x7f946c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.116+0000 7f947f7fe640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 0x7f946c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:50:09.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.134+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f94801a26e0 con 0x7f9480101c30 2024-09-12T02:50:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.185+0000 7f947cff9640 1 -- 172.21.15.46:0/764146336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f94700c0730 con 0x7f9480101c30 2024-09-12T02:50:09.187 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:08.985+0000 7f206f067040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:50:09.188 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:09.068+0000 7f206f067040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:09.188 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:09.126+0000 7f206f067040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:50:09.188 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:09.186+0000 7f206f067040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:50:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.188+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 msgr2=0x7f946c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.188+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 0x7f946c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.188+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 msgr2=0x7f948019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.188+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9470005940 tx=0x7f94700042f0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 shutdown_connections 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94801025f0 0x7f94801a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480101c30 0x7f948019a440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f946c0a2c80 0x7f946c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 --2- 172.21.15.46:0/764146336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9480100a40 0x7f9480199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 >> 172.21.15.46:0/764146336 conn(0x7f94800fc1e0 msgr2=0x7f94800698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 shutdown_connections 2024-09-12T02:50:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.189+0000 7f948615b640 1 -- 172.21.15.46:0/764146336 wait complete. 2024-09-12T02:50:09.194 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.932+0000 7f3e905b3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:50:09.194 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:08.998+0000 7f3e905b3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:50:09.195 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:09.081+0000 7f3e905b3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:09.195 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:09.139+0000 7f3e905b3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:50:09.201 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T02:50:09.213 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T02:50:09.225 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:50:09.226 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:50:09.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T02:50:09.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:50:09.249 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.021+0000 7fe2abb66040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:50:09.249 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.190+0000 7fe2abb66040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 -- 172.21.15.46:0/2354309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c107820 msgr2=0x7eff2c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 --2- 172.21.15.46:0/2354309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c107820 0x7eff2c109c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7eff24009ca0 tx=0x7eff2402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 -- 172.21.15.46:0/2354309768 shutdown_connections 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 --2- 172.21.15.46:0/2354309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c107820 0x7eff2c109c90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 --2- 172.21.15.46:0/2354309768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 0x7eff2c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 --2- 172.21.15.46:0/2354309768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c1025c0 0x7eff2c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 -- 172.21.15.46:0/2354309768 >> 172.21.15.46:0/2354309768 conn(0x7eff2c0fc1e0 msgr2=0x7eff2c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 -- 172.21.15.46:0/2354309768 shutdown_connections 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.334+0000 7eff32b49640 1 -- 172.21.15.46:0/2354309768 wait complete. 2024-09-12T02:50:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 Processor -- start 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 -- start start 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 0x7eff2c100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c107820 0x7eff2c100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10d6b0 con 0x7eff2c1025c0 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10d530 con 0x7eff2c104ef0 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff32b49640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10d830 con 0x7eff2c107820 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52940/0 (socket says 172.21.15.46:52940) 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 -- 172.21.15.46:0/2615805632 learned_addr learned my addr 172.21.15.46:0/2615805632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff310e6640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c107820 0x7eff2c100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff2bfff640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 0x7eff2c100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c107820 msgr2=0x7eff2c100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c107820 0x7eff2c100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 msgr2=0x7eff2c100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.335+0000 7eff308e5640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 0x7eff2c100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff308e5640 1 -- 172.21.15.46:0/2615805632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c101340 con 0x7eff2c1025c0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff308e5640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff2000e9f0 tx=0x7eff2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2000cd80 con 0x7eff2c1025c0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1a1e60 con 0x7eff2c1025c0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff200054d0 con 0x7eff2c1025c0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20010490 con 0x7eff2c1025c0 2024-09-12T02:50:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.336+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a2260 con 0x7eff2c1025c0 2024-09-12T02:50:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.338+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c0fdcd0 con 0x7eff2c1025c0 2024-09-12T02:50:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.338+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7eff20005020 con 0x7eff2c1025c0 2024-09-12T02:50:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.339+0000 7eff29ffb640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 0x7eff1c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.339+0000 7eff2bfff640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 msgr2=0x7eff1c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.339+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7eff200cf760 con 0x7eff2c1025c0 2024-09-12T02:50:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.339+0000 7eff2bfff640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 0x7eff1c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:09.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.343+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff2009c340 con 0x7eff2c1025c0 2024-09-12T02:50:09.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:09 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:50:09.532 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.262+0000 7fe2abb66040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:50:09.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.338+0000 7fe2abb66040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:50:09.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.428+0000 7fe2abb66040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:09.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.492+0000 7fe2abb66040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:50:09.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.540+0000 7eff2bfff640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 msgr2=0x7eff1c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:09.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.540+0000 7eff2bfff640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 0x7eff1c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:50:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.549+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7eff2c062f00 con 0x7eff2c1025c0 2024-09-12T02:50:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.553+0000 7eff29ffb640 1 -- 172.21.15.46:0/2615805632 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7eff2009bc40 con 0x7eff2c1025c0 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 msgr2=0x7eff1c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 0x7eff1c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 msgr2=0x7eff2c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff2000e9f0 tx=0x7eff2000edf0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 shutdown_connections 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7eff1c0a2c80 0x7eff1c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff2c107820 0x7eff2c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c104ef0 0x7eff2c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 --2- 172.21.15.46:0/2615805632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c1025c0 0x7eff2c101f30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 >> 172.21.15.46:0/2615805632 conn(0x7eff2c0fc1e0 msgr2=0x7eff2c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 shutdown_connections 2024-09-12T02:50:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.555+0000 7eff32b49640 1 -- 172.21.15.46:0/2615805632 wait complete. 2024-09-12T02:50:09.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T02:50:09.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:09.200+0000 7f3e905b3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:50:09.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T02:50:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 -- 172.21.15.46:0/795529522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8102d50 msgr2=0x7efff81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 --2- 172.21.15.46:0/795529522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8102d50 0x7efff81031d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effec009900 tx=0x7effec02a410 comp rx=0 tx=0).stop 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 -- 172.21.15.46:0/795529522 shutdown_connections 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 --2- 172.21.15.46:0/795529522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103710 0x7efff8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 --2- 172.21.15.46:0/795529522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8102d50 0x7efff81031d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 --2- 172.21.15.46:0/795529522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8101b60 0x7efff8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 -- 172.21.15.46:0/795529522 >> 172.21.15.46:0/795529522 conn(0x7efff80fd2c0 msgr2=0x7efff80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 -- 172.21.15.46:0/795529522 shutdown_connections 2024-09-12T02:50:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 -- 172.21.15.46:0/795529522 wait complete. 2024-09-12T02:50:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.716+0000 7efffe79f640 1 Processor -- start 2024-09-12T02:50:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 -- start start 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 0x7efff8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 0x7efff80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efff810d670 con 0x7efff8103710 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efff810d4f0 con 0x7efff8102d50 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffe79f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efff810d7f0 con 0x7efff8101b60 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffcd3c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 0x7efff8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffcd3c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52960/0 (socket says 172.21.15.46:52960) 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff77fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 0x7efff80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efffcd3c640 1 -- 172.21.15.46:0/591116641 learned_addr learned my addr 172.21.15.46:0/591116641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 -- 172.21.15.46:0/591116641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 msgr2=0x7efff80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 0x7efff80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 -- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 msgr2=0x7efff8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.717+0000 7efff7fff640 1 -- 172.21.15.46:0/591116641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff80753b0 con 0x7efff8101b60 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efff77fe640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 0x7efff80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efffcd3c640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efff7fff640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 0x7efff8075fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effe800e9c0 tx=0x7effe800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe800ccc0 con 0x7efff8101b60 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81a1ea0 con 0x7efff8101b60 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effe80035f0 con 0x7efff8101b60 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe800ce60 con 0x7efff8101b60 2024-09-12T02:50:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.718+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a22b0 con 0x7efff8101b60 2024-09-12T02:50:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.719+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff8108f20 con 0x7efff8101b60 2024-09-12T02:50:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.720+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7effe8004020 con 0x7efff8101b60 2024-09-12T02:50:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.721+0000 7efff57fa640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.721+0000 7efff77fe640 1 -- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 msgr2=0x7effe00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:09.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.721+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7effe80cf870 con 0x7efff8101b60 2024-09-12T02:50:09.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.721+0000 7efff77fe640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:09.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.725+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effe809c3f0 con 0x7efff8101b60 2024-09-12T02:50:09.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.907+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7efff8062e20 con 0x7efff8101b60 2024-09-12T02:50:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.914+0000 7efff57fa640 1 -- 172.21.15.46:0/591116641 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7effe809bcf0 con 0x7efff8101b60 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 msgr2=0x7effe00a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 msgr2=0x7efff8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 0x7efff8075fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effe800e9c0 tx=0x7effe800ee80 comp rx=0 tx=0).stop 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 shutdown_connections 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.916+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff8103710 0x7efff8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efff8102d50 0x7efff80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 --2- 172.21.15.46:0/591116641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8101b60 0x7efff8075fa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 >> 172.21.15.46:0/591116641 conn(0x7efff80fd2c0 msgr2=0x7efff81060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 shutdown_connections 2024-09-12T02:50:09.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:09.917+0000 7efffe79f640 1 -- 172.21.15.46:0/591116641 wait complete. 2024-09-12T02:50:09.927 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:09 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:09.558+0000 7fe2abb66040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:50:09.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T02:50:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 -- 172.21.15.46:0/1773037368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf341078a0 msgr2=0x7faf34109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 --2- 172.21.15.46:0/1773037368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf341078a0 0x7faf34109c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009ca0 tx=0x7faf2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 -- 172.21.15.46:0/1773037368 shutdown_connections 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 --2- 172.21.15.46:0/1773037368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf341078a0 0x7faf34109c90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 --2- 172.21.15.46:0/1773037368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf34104f70 0x7faf34107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 --2- 172.21.15.46:0/1773037368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 -- 172.21.15.46:0/1773037368 >> 172.21.15.46:0/1773037368 conn(0x7faf340fc1e0 msgr2=0x7faf340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 -- 172.21.15.46:0/1773037368 shutdown_connections 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.036+0000 7faf38f66640 1 -- 172.21.15.46:0/1773037368 wait complete. 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 Processor -- start 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 -- start start 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 0x7faf34198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf341078a0 0x7faf3419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410d690 con 0x7faf34104f70 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410d510 con 0x7faf341078a0 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.037+0000 7faf38f66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410d810 con 0x7faf34102640 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf31d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 0x7faf34198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf31d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 0x7faf34198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52984/0 (socket says 172.21.15.46:52984) 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf31d9b640 1 -- 172.21.15.46:0/2481246588 learned_addr learned my addr 172.21.15.46:0/2481246588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf32d9d640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf341078a0 0x7faf3419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 -- 172.21.15.46:0/2481246588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf341078a0 msgr2=0x7faf3419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf341078a0 0x7faf3419f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 -- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 msgr2=0x7faf34198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 0x7faf34198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 -- 172.21.15.46:0/2481246588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3419f920 con 0x7faf34102640 2024-09-12T02:50:10.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf3259c640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34197d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e9f0 tx=0x7faf2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2400ccc0 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf240108e0 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3419fc10 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.038+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24018880 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.039+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a0160 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.039+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf340fdcd0 con 0x7faf34102640 2024-09-12T02:50:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.041+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7faf24010a90 con 0x7faf34102640 2024-09-12T02:50:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.042+0000 7faf237fe640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 0x7faf1c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.042+0000 7faf31d9b640 1 -- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 msgr2=0x7faf1c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.042+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7faf240cfb20 con 0x7faf34102640 2024-09-12T02:50:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.042+0000 7faf31d9b640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 0x7faf1c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.044+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf2409c0b0 con 0x7faf34102640 2024-09-12T02:50:10.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.224+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7faf34062e90 con 0x7faf34102640 2024-09-12T02:50:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.229+0000 7faf237fe640 1 -- 172.21.15.46:0/2481246588 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7faf240a03b0 con 0x7faf34102640 2024-09-12T02:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.231+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 msgr2=0x7faf1c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.231+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 0x7faf1c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.231+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 msgr2=0x7faf34197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.231+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34197d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e9f0 tx=0x7faf2400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 shutdown_connections 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf341078a0 0x7faf3419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34104f70 0x7faf34198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7faf1c0a2c80 0x7faf1c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 --2- 172.21.15.46:0/2481246588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf34102640 0x7faf34197d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 >> 172.21.15.46:0/2481246588 conn(0x7faf340fc1e0 msgr2=0x7faf34102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 shutdown_connections 2024-09-12T02:50:10.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.232+0000 7faf38f66640 1 -- 172.21.15.46:0/2481246588 wait complete. 2024-09-12T02:50:10.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T02:50:10.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 -- 172.21.15.46:0/1402529392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35801078c0 msgr2=0x7f3580109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 --2- 172.21.15.46:0/1402529392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35801078c0 0x7f3580109cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3578009ca0 tx=0x7f357802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 -- 172.21.15.46:0/1402529392 shutdown_connections 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 --2- 172.21.15.46:0/1402529392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35801078c0 0x7f3580109cb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 --2- 172.21.15.46:0/1402529392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3580104f90 0x7f3580107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 --2- 172.21.15.46:0/1402529392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3580102660 0x7f3580104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 -- 172.21.15.46:0/1402529392 >> 172.21.15.46:0/1402529392 conn(0x7f35800fc1e0 msgr2=0x7f35800fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 -- 172.21.15.46:0/1402529392 shutdown_connections 2024-09-12T02:50:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.351+0000 7f35873b7640 1 -- 172.21.15.46:0/1402529392 wait complete. 2024-09-12T02:50:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 Processor -- start 2024-09-12T02:50:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 -- start start 2024-09-12T02:50:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3580102660 0x7f3580101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 0x7f3580100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35801078c0 0x7f3580100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f358010d690 con 0x7f3580104f90 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f358010d510 con 0x7f3580102660 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f35873b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f358010d810 con 0x7f35801078c0 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 0x7f3580100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 0x7f3580100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52994/0 (socket says 172.21.15.46:52994) 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 -- 172.21.15.46:0/2855543716 learned_addr learned my addr 172.21.15.46:0/2855543716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3585153640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3580102660 0x7f3580101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3585954640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35801078c0 0x7f3580100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35801078c0 msgr2=0x7f3580100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35801078c0 0x7f3580100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3580102660 msgr2=0x7f3580101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3580102660 0x7f3580101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.352+0000 7f3584952640 1 -- 172.21.15.46:0/2855543716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35801013e0 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f3584952640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 0x7f3580100610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9c0 tx=0x7f357000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357000cd80 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35700054d0 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801a1e30 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570010430 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.353+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801a2290 con 0x7f3580104f90 2024-09-12T02:50:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.354+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35800fdcd0 con 0x7f3580104f90 2024-09-12T02:50:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.356+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f3570004fb0 con 0x7f3580104f90 2024-09-12T02:50:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.356+0000 7f35767fc640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f356c0a2c80 0x7f356c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.356+0000 7f3585153640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f356c0a2c80 msgr2=0x7f356c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3748867549 2024-09-12T02:50:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.356+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9121+0+0 (secure 0 0 0) 0x7f35700cfa00 con 0x7f3580104f90 2024-09-12T02:50:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.357+0000 7f3585153640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f356c0a2c80 0x7f356c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 02:50:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y[36043]: 2024-09-12T02:50:09.989+0000 7f3e905b3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2615805632' entity='client.admin' 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='client.? ' entity='client.admin' 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: Standby manager daemon z restarted 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: Standby manager daemon z started 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: Standby manager daemon y restarted 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: Standby manager daemon y started 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:10.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:10 smithi145 ceph-mon[34674]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.364+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35700541b0 con 0x7f3580104f90 2024-09-12T02:50:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.444+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f3570098550 con 0x7f3580104f90 2024-09-12T02:50:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.445+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f356c0a2c80 msgr2=0x7f356c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:10.445+0000 7f35767fc640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/3748867549,v1:172.21.15.46:6801/3748867549] conn(0x7f356c0a2c80 0x7f356c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.470 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 02:50:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z[36392]: 2024-09-12T02:50:09.979+0000 7f206f067040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2615805632' entity='client.admin' 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: Standby manager daemon z restarted 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: Standby manager daemon z started 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: Standby manager daemon y restarted 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: Standby manager daemon y started 2024-09-12T02:50:10.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:10.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:10.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:10 smithi196 ceph-mon[34587]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:10.533 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.428+0000 7fe2abb66040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:50:10.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/764146336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T02:50:10.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:50:10.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2615805632' entity='client.admin' 2024-09-12T02:50:10.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='client.? ' entity='client.admin' 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: Standby manager daemon z restarted 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: Standby manager daemon z started 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.196:0/3858272295' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: Standby manager daemon y restarted 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: Standby manager daemon y started 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:10.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-mon[30951]: from='mgr.? 172.21.15.145:0/1129596997' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.743+0000 7fe212176640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.743+0000 7fe212176640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.743+0000 7fe212176640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.743+0000 7fe212176640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.743+0000 7fe212176640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.756+0000 7fe21497b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.034 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.756+0000 7fe21497b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.034 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.756+0000 7fe21497b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.034 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.756+0000 7fe21497b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.034 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:10 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:10.756+0000 7fe21497b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:50:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.448+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f35700a46c0 con 0x7f3580104f90 2024-09-12T02:50:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.449+0000 7f35767fc640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f356c0518a0 0x7f356c051c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.449+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f356c00c450 con 0x7f356c0518a0 2024-09-12T02:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.452+0000 7f3585153640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f356c0518a0 0x7f356c051c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:11.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.455+0000 7f3585153640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f356c0518a0 0x7f356c051c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f357c0097f0 tx=0x7f357c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.514+0000 7f35767fc640 1 -- 172.21.15.46:0/2855543716 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f356c00c450 con 0x7f356c0518a0 2024-09-12T02:50:11.515 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path / 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.516+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f356c0518a0 msgr2=0x7f356c051c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f356c0518a0 0x7f356c051c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f357c0097f0 tx=0x7f357c005f90 comp rx=0 tx=0).stop 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 msgr2=0x7f3580100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 --2- 172.21.15.46:0/2855543716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3580104f90 0x7f3580100610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9c0 tx=0x7f357000ee80 comp rx=0 tx=0).stop 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f3585954640 1 -- 172.21.15.46:0/2855543716 reap_dead start 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 shutdown_connections 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 >> 172.21.15.46:0/2855543716 conn(0x7f35800fc1e0 msgr2=0x7f3580103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 shutdown_connections 2024-09-12T02:50:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.517+0000 7f35873b7640 1 -- 172.21.15.46:0/2855543716 wait complete. 2024-09-12T02:50:11.529 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='client.? ' entity='client.admin' 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: Active manager daemon x restarted 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: Activating manager daemon x 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: mgrmap e20: x(active, starting, since 0.0138895s), standbys: z, y 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:50:11.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:50:11.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: Manager daemon x is now available 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:50:11.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:11 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:50:11.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T02:50:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='client.? ' entity='client.admin' 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: Active manager daemon x restarted 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: Activating manager daemon x 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: mgrmap e20: x(active, starting, since 0.0138895s), standbys: z, y 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:50:11.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:50:11.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: Manager daemon x is now available 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:50:11.614 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:11 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:50:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 -- 172.21.15.46:0/754464720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41025f0 msgr2=0x7ff9c4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/754464720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41025f0 0x7ff9c4108e80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009ca0 tx=0x7ff9bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 -- 172.21.15.46:0/754464720 shutdown_connections 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/754464720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41025f0 0x7ff9c4108e80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/754464720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 0x7ff9c41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/754464720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c4100a40 0x7ff9c4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 -- 172.21.15.46:0/754464720 >> 172.21.15.46:0/754464720 conn(0x7ff9c40fc1e0 msgr2=0x7ff9c40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 -- 172.21.15.46:0/754464720 shutdown_connections 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 -- 172.21.15.46:0/754464720 wait complete. 2024-09-12T02:50:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.645+0000 7ff9cacd9640 1 Processor -- start 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 -- start start 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 0x7ff9c4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 0x7ff9c419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410c570 con 0x7ff9c4100a40 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410c3f0 con 0x7ff9c4101c30 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410c6f0 con 0x7ff9c41025f0 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53090/0 (socket says 172.21.15.46:53090) 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 -- 172.21.15.46:0/1305320483 learned_addr learned my addr 172.21.15.46:0/1305320483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c9276640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 0x7ff9c419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 -- 172.21.15.46:0/1305320483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 msgr2=0x7ff9c419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 0x7ff9c419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 -- 172.21.15.46:0/1305320483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 msgr2=0x7ff9c4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c3fff640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 0x7ff9c4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 0x7ff9c4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 -- 172.21.15.46:0/1305320483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c419f940 con 0x7ff9c4100a40 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c9276640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 0x7ff9c419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c8a75640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9c0 tx=0x7ff9b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800cd80 con 0x7ff9c4100a40 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9b80035f0 con 0x7ff9c4100a40 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.646+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c419fc30 con 0x7ff9c4100a40 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.647+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8010690 con 0x7ff9c4100a40 2024-09-12T02:50:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.647+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a0130 con 0x7ff9c4100a40 2024-09-12T02:50:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.648+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c41a0360 con 0x7ff9c4100a40 2024-09-12T02:50:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.649+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7ff9b80037a0 con 0x7ff9c4100a40 2024-09-12T02:50:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.649+0000 7ff9c1ffb640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 0x7ff9b40a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.649+0000 7ff9c3fff640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 0x7ff9b40a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.649+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff9b80cf810 con 0x7ff9c4100a40 2024-09-12T02:50:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.650+0000 7ff9c3fff640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 0x7ff9b40a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00095f0 tx=0x7ff9b0005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.654+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9b809c5e0 con 0x7ff9c4100a40 2024-09-12T02:50:11.712 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-12T02:50:11.712 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: Active manager daemon x restarted 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: Activating manager daemon x 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: mgrmap e20: x(active, starting, since 0.0138895s), standbys: z, y 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:50:11.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:11.714 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:50:11.715 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: Manager daemon x is now available 2024-09-12T02:50:11.716 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.716 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:50:11.716 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:11 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:50:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.871+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff9c40fe3f0 con 0x7ff9b40a2a40 2024-09-12T02:50:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.927+0000 7ff9c1ffb640 1 -- 172.21.15.46:0/1305320483 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff9c40fe3f0 con 0x7ff9b40a2a40 2024-09-12T02:50:11.927 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path / 2024-09-12T02:50:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 msgr2=0x7ff9b40a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 0x7ff9b40a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00095f0 tx=0x7ff9b0005f90 comp rx=0 tx=0).stop 2024-09-12T02:50:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 msgr2=0x7ff9c4197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9c0 tx=0x7ff9b800ee80 comp rx=0 tx=0).stop 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 shutdown_connections 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff9b40a2a40 0x7ff9b40a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c41025f0 0x7ff9c419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9c4101c30 0x7ff9c4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 --2- 172.21.15.46:0/1305320483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a40 0x7ff9c4197d30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 >> 172.21.15.46:0/1305320483 conn(0x7ff9c40fc1e0 msgr2=0x7ff9c40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 shutdown_connections 2024-09-12T02:50:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:11.929+0000 7ff9cacd9640 1 -- 172.21.15.46:0/1305320483 wait complete. 2024-09-12T02:50:11.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 -- 172.21.15.46:0/2738779966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1874104ef0 msgr2=0x7f18741072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 --2- 172.21.15.46:0/2738779966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1874104ef0 0x7f18741072e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c02a410 comp rx=0 tx=0).stop 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 -- 172.21.15.46:0/2738779966 shutdown_connections 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 --2- 172.21.15.46:0/2738779966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874107820 0x7f1874109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 --2- 172.21.15.46:0/2738779966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1874104ef0 0x7f18741072e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 --2- 172.21.15.46:0/2738779966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18741025c0 0x7f18741049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 -- 172.21.15.46:0/2738779966 >> 172.21.15.46:0/2738779966 conn(0x7f18740fc1e0 msgr2=0x7f18740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 -- 172.21.15.46:0/2738779966 shutdown_connections 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.045+0000 7f187995f640 1 -- 172.21.15.46:0/2738779966 wait complete. 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 Processor -- start 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 -- start start 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874104ef0 0x7f18741982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874107820 0x7f187419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d6a0 con 0x7f18741025c0 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d520 con 0x7f1874104ef0 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f187995f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f187410d820 con 0x7f1874107820 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53108/0 (socket says 172.21.15.46:53108) 2024-09-12T02:50:12.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f1872ffd640 1 -- 172.21.15.46:0/2356004771 learned_addr learned my addr 172.21.15.46:0/2356004771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f18727fc640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874104ef0 0x7f18741982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.046+0000 7f18737fe640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874107820 0x7f187419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 -- 172.21.15.46:0/2356004771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874107820 msgr2=0x7f187419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874107820 0x7f187419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 -- 172.21.15.46:0/2356004771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874104ef0 msgr2=0x7f18741982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874104ef0 0x7f18741982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 -- 172.21.15.46:0/2356004771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187419f9b0 con 0x7f18741025c0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f1872ffd640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f186400eb90 tx=0x7f186400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864004d20 con 0x7f18741025c0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f186400ee90 con 0x7f18741025c0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187419fca0 con 0x7f18741025c0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864010670 con 0x7f18741025c0 2024-09-12T02:50:12.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.047+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a01a0 con 0x7f18741025c0 2024-09-12T02:50:12.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.048+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18740fdcd0 con 0x7f18741025c0 2024-09-12T02:50:12.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.049+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f1864005020 con 0x7f18741025c0 2024-09-12T02:50:12.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.050+0000 7f187895d640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 0x7f18600a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.050+0000 7f18727fc640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 0x7f18600a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.050+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f18640cf7c0 con 0x7f18741025c0 2024-09-12T02:50:12.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.051+0000 7f18727fc640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 0x7f18600a4f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f185c004770 tx=0x7f185c000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:12.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.054+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f186409c590 con 0x7f18741025c0 2024-09-12T02:50:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.271+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1874062630 con 0x7f18600a2a40 2024-09-12T02:50:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.278+0000 7f187895d640 1 -- 172.21.15.46:0/2356004771 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1874062630 con 0x7f18600a2a40 2024-09-12T02:50:12.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T02:50:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T02:50:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 msgr2=0x7f18600a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 0x7f18600a4f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f185c004770 tx=0x7f185c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 msgr2=0x7f1874197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f186400eb90 tx=0x7f186400c6a0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 shutdown_connections 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18600a2a40 0x7f18600a4f00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1874107820 0x7f187419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1874104ef0 0x7f18741982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 --2- 172.21.15.46:0/2356004771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18741025c0 0x7f1874197da0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 >> 172.21.15.46:0/2356004771 conn(0x7f18740fc1e0 msgr2=0x7f1874102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 shutdown_connections 2024-09-12T02:50:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.281+0000 7f187995f640 1 -- 172.21.15.46:0/2356004771 wait complete. 2024-09-12T02:50:12.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 -- 172.21.15.46:0/176115587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 msgr2=0x7f575c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 --2- 172.21.15.46:0/176115587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c102100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5748009900 tx=0x7f574802a480 comp rx=0 tx=0).stop 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 -- 172.21.15.46:0/176115587 shutdown_connections 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 --2- 172.21.15.46:0/176115587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c102640 0x7f575c108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 --2- 172.21.15.46:0/176115587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c102100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 --2- 172.21.15.46:0/176115587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c100f70 0x7f575c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.408+0000 7f5762dbd640 1 -- 172.21.15.46:0/176115587 >> 172.21.15.46:0/176115587 conn(0x7f575c0fb970 msgr2=0x7f575c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- 172.21.15.46:0/176115587 shutdown_connections 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- 172.21.15.46:0/176115587 wait complete. 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 Processor -- start 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- start start 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c100f70 0x7f575c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 0x7f575c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10c580 con 0x7f575c101c80 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10c400 con 0x7f575c102640 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.409+0000 7f5762dbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10c700 con 0x7f575c100f70 2024-09-12T02:50:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53118/0 (socket says 172.21.15.46:53118) 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 -- 172.21.15.46:0/221399302 learned_addr learned my addr 172.21.15.46:0/221399302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f576135a640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 0x7f575c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f5760b59640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c100f70 0x7f575c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 -- 172.21.15.46:0/221399302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c100f70 msgr2=0x7f575c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c100f70 0x7f575c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 -- 172.21.15.46:0/221399302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 msgr2=0x7f575c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 0x7f575c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 -- 172.21.15.46:0/221399302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c19f930 con 0x7f575c101c80 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f576135a640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 0x7f575c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f575bfff640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5748005940 tx=0x7f57480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574803f030 con 0x7f575c101c80 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f5762dbd640 1 -- 172.21.15.46:0/221399302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c19fbc0 con 0x7f575c101c80 2024-09-12T02:50:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5748003cb0 con 0x7f575c101c80 2024-09-12T02:50:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.410+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748035070 con 0x7f575c101c80 2024-09-12T02:50:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.411+0000 7f5762dbd640 1 -- 172.21.15.46:0/221399302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c1a0100 con 0x7f575c101c80 2024-09-12T02:50:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.412+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c062df0 con 0x7f575c101c80 2024-09-12T02:50:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.413+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f5748042030 con 0x7f575c101c80 2024-09-12T02:50:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.414+0000 7f5759ffb640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 0x7f574c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.414+0000 7f5760b59640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 0x7f574c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.414+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f57480ef640 con 0x7f575c101c80 2024-09-12T02:50:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.414+0000 7f5760b59640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 0x7f574c0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f575c069ef0 tx=0x7f5750005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.418+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57480bbd80 con 0x7f575c101c80 2024-09-12T02:50:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.461+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f57480b83f0 con 0x7f575c101c80 2024-09-12T02:50:12.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.671+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f575c062630 con 0x7f574c0a2a40 2024-09-12T02:50:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.678+0000 7f5759ffb640 1 -- 172.21.15.46:0/221399302 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f575c062630 con 0x7f574c0a2a40 2024-09-12T02:50:12.678 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-09-12T02:50:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 msgr2=0x7f574c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 0x7f574c0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f575c069ef0 tx=0x7f5750005f90 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 msgr2=0x7f575c19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5748005940 tx=0x7f57480042f0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 shutdown_connections 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f574c0a2a40 0x7f574c0a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f575c102640 0x7f575c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f575c101c80 0x7f575c19ca20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 --2- 172.21.15.46:0/221399302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f575c100f70 0x7f575c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 >> 172.21.15.46:0/221399302 conn(0x7f575c0fb970 msgr2=0x7f575c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 shutdown_connections 2024-09-12T02:50:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T02:50:12.680+0000 7f57477fe640 1 -- 172.21.15.46:0/221399302 wait complete. 2024-09-12T02:50:12.690 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:12 smithi046 ceph-mon[30951]: mgrmap e21: x(active, since 1.02029s), standbys: z, y 2024-09-12T02:50:12.690 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:12 smithi046 ceph-mon[30951]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.690 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:12 smithi046 ceph-mon[30951]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:12.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:12 smithi046 ceph-mon[30951]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T02:50:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:12 smithi196 ceph-mon[34587]: mgrmap e21: x(active, since 1.02029s), standbys: z, y 2024-09-12T02:50:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:12 smithi196 ceph-mon[34587]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:12 smithi196 ceph-mon[34587]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:12 smithi196 ceph-mon[34587]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.722 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T02:50:12.733 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T02:50:12.748 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T02:50:12.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T02:50:12.767 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T02:50:12.785 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:50:12.786 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T02:50:12.786 INFO:tasks.workunit:timeout=3h 2024-09-12T02:50:12.786 INFO:tasks.workunit:cleanup=True 2024-09-12T02:50:12.786 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:12.803 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:12.803 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-12 02:48:25.734216024 +0000 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-12 02:48:25.734216024 +0000 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-12 02:50:08.760319545 +0000 2024-09-12T02:50:12.804 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-12T02:50:12.804 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T02:50:12.804 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T02:50:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:12 smithi145 ceph-mon[34674]: mgrmap e21: x(active, since 1.02029s), standbys: z, y 2024-09-12T02:50:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:12 smithi145 ceph-mon[34674]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:12 smithi145 ceph-mon[34674]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:12 smithi145 ceph-mon[34674]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:12.876 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:50:12.935 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T02:50:13.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:13 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:13.785 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:13.570+0000 7fe1fab08640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:50:13] ENGINE Error in HTTPServer.serve 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: Traceback (most recent call last): 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._connections.run(self.expiration_interval) 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._run(expiration_interval) 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: s = self.context.wrap_socket( 2024-09-12T02:50:13.786 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: return self.sslsocket_class._create( 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self.do_handshake() 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: self._sslobj.do_handshake() 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:50:13.787 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:50:13 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:50:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:13.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:13.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:13 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: [12/Sep/2024:02:50:13] ENGINE Bus STARTING 2024-09-12T02:50:14.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: [12/Sep/2024:02:50:13] ENGINE Serving on https://172.21.15.46:7150 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: [12/Sep/2024:02:50:13] ENGINE Error in HTTPServer.serve 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: Traceback (most recent call last): 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._run(expiration_interval) 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: s = self.context.wrap_socket( 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:50:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout: return self.sslsocket_class._create( 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout: self.do_handshake() 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._sslobj.do_handshake() 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: [12/Sep/2024:02:50:13] ENGINE Serving on http://172.21.15.46:8765 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: [12/Sep/2024:02:50:13] ENGINE Bus STARTED 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:14 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T02:50:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T02:50:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T02:50:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T02:50:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T02:50:14.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: [12/Sep/2024:02:50:13] ENGINE Bus STARTING 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: [12/Sep/2024:02:50:13] ENGINE Serving on https://172.21.15.46:7150 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: [12/Sep/2024:02:50:13] ENGINE Error in HTTPServer.serve 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:50:14.785 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: [12/Sep/2024:02:50:13] ENGINE Serving on http://172.21.15.46:8765 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: [12/Sep/2024:02:50:13] ENGINE Bus STARTED 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:14 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: [12/Sep/2024:02:50:13] ENGINE Bus STARTING 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: [12/Sep/2024:02:50:13] ENGINE Serving on https://172.21.15.46:7150 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: [12/Sep/2024:02:50:13] ENGINE Error in HTTPServer.serve 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout: Traceback (most recent call last): 2024-09-12T02:50:14.861 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._run(expiration_interval) 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: s = self.context.wrap_socket( 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: return self.sslsocket_class._create( 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: self.do_handshake() 2024-09-12T02:50:14.862 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._sslobj.do_handshake() 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: [12/Sep/2024:02:50:13] ENGINE Serving on http://172.21.15.46:8765 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: [12/Sep/2024:02:50:13] ENGINE Bus STARTED 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.863 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: Updating smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:14.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:14 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:15.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:15.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:15.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:15 smithi196 ceph-mon[34587]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:15.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:15 smithi046 ceph-mon[30951]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-12T02:50:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:50:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:50:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:15 smithi145 ceph-mon[34674]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-12T02:50:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:17 smithi196 ceph-mon[34587]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-09-12T02:50:17.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:17 smithi046 ceph-mon[30951]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-09-12T02:50:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:17 smithi145 ceph-mon[34674]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-09-12T02:50:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:19 smithi196 ceph-mon[34587]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T02:50:19.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:19 smithi046 ceph-mon[30951]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T02:50:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:19 smithi145 ceph-mon[34674]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T02:50:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:21 smithi196 ceph-mon[34587]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T02:50:21.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:21 smithi046 ceph-mon[30951]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T02:50:21.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:21 smithi145 ceph-mon[34674]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T02:50:23.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:23 smithi196 ceph-mon[34587]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T02:50:23.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:23 smithi046 ceph-mon[30951]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T02:50:23.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:23 smithi145 ceph-mon[34674]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T02:50:25.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:25 smithi196 ceph-mon[34587]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T02:50:25.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:25 smithi046 ceph-mon[30951]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T02:50:25.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:25 smithi145 ceph-mon[34674]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 -- 172.21.15.46:0/737299194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 msgr2=0x7fcde4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/737299194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde4100e30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009930 tx=0x7fcdd402a420 comp rx=0 tx=0).stop 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 -- 172.21.15.46:0/737299194 shutdown_connections 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/737299194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde4101370 0x7fcde410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/737299194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde4100e30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/737299194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde41069e0 0x7fcde4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.910+0000 7fcdeb35b640 1 -- 172.21.15.46:0/737299194 >> 172.21.15.46:0/737299194 conn(0x7fcde40fc6f0 msgr2=0x7fcde40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- 172.21.15.46:0/737299194 shutdown_connections 2024-09-12T02:50:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- 172.21.15.46:0/737299194 wait complete. 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 Processor -- start 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- start start 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 0x7fcde4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde41069e0 0x7fcde419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4110cd0 con 0x7fcde41009d0 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4110b50 con 0x7fcde4101370 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.911+0000 7fcdeb35b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4110e50 con 0x7fcde41069e0 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51280/0 (socket says 172.21.15.46:51280) 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 -- 172.21.15.46:0/1079918799 learned_addr learned my addr 172.21.15.46:0/1079918799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde98f8640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde41069e0 0x7fcde419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde88f6640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 0x7fcde4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 -- 172.21.15.46:0/1079918799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde41069e0 msgr2=0x7fcde419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde41069e0 0x7fcde419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 -- 172.21.15.46:0/1079918799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 msgr2=0x7fcde4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 0x7fcde4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 -- 172.21.15.46:0/1079918799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde419c1b0 con 0x7fcde41009d0 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde88f6640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 0x7fcde4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcde90f7640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e9f0 tx=0x7fcde000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcdda7fc640 1 -- 172.21.15.46:0/1079918799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde000cd80 con 0x7fcde41009d0 2024-09-12T02:50:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcdda7fc640 1 -- 172.21.15.46:0/1079918799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde00108e0 con 0x7fcde41009d0 2024-09-12T02:50:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcdda7fc640 1 -- 172.21.15.46:0/1079918799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0018920 con 0x7fcde41009d0 2024-09-12T02:50:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde419c4a0 con 0x7fcde41009d0 2024-09-12T02:50:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.912+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde41a3d30 con 0x7fcde41009d0 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.915+0000 7fcdda7fc640 1 -- 172.21.15.46:0/1079918799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fcde0010a90 con 0x7fcde41009d0 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.915+0000 7fcdee1cf740 1 Processor -- start 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.915+0000 7fcdee1cf740 1 -- start start 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec09fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564aec0042f0 con 0x564aec0a02d0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564aebfac270 con 0x564aec09f9b0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564aec0002b0 con 0x564aec0acec0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51284/0 (socket says 172.21.15.46:51284) 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 -- 172.21.15.46:0/2968446679 learned_addr learned my addr 172.21.15.46:0/2968446679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde98f8640 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec09fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdda7fc640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 0x7fcdd00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde88f6640 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde98f8640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 0x7fcdd00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 -- 172.21.15.46:0/2968446679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 msgr2=0x564aec0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 -- 172.21.15.46:0/2968446679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 msgr2=0x564aec09fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec09fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcde90f7640 1 -- 172.21.15.46:0/2968446679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aebfab340 con 0x564aec0a02d0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.916+0000 7fcdda7fc640 1 -- 172.21.15.46:0/1079918799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fcde00cfcf0 con 0x7fcde41009d0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.917+0000 7fcde90f7640 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a48c0 tx=0x7fcde0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34277ba347af208f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.917+0000 7fcde98f8640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 0x7fcdd00a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009370 tx=0x7fcddc004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.917+0000 7fcdcf7fe640 1 -- 172.21.15.46:0/2968446679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0010de0 con 0x564aec0a02d0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.917+0000 7fcdcf7fe640 1 -- 172.21.15.46:0/2968446679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde009c3e0 con 0x564aec0a02d0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.917+0000 7fcdcf7fe640 1 -- 172.21.15.46:0/2968446679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00d32f0 con 0x564aec0a02d0 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2968446679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 msgr2=0x564aec0ac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a48c0 tx=0x7fcde0024000 comp rx=0 tx=0).stop 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2968446679 shutdown_connections 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0ac980 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2968446679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec09fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2968446679 >> 172.21.15.46:0/2968446679 conn(0x564aec09c060 msgr2=0x564aec09c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2968446679 shutdown_connections 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2968446679 wait complete. 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 -- start start 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.918+0000 7fcdee1cf740 1 Processor -- start 2024-09-12T02:50:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0aa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564aec0afa30 con 0x564aec0a02d0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564aebf8d290 con 0x564aec09f9b0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564aec0afbb0 con 0x564aec0acec0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 10 client.0 _ll_register_callbacks cb 0x564aebfbe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51296/0 (socket says 172.21.15.46:51296) 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 -- 172.21.15.46:0/2180413425 learned_addr learned my addr 172.21.15.46:0/2180413425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde98f8640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 -- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 msgr2=0x564aec0aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 -- 172.21.15.46:0/2180413425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 msgr2=0x564aec0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 -- 172.21.15.46:0/2180413425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aebd17210 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde98f8640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcde88f6640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009900 tx=0x7fcdd402aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c305845e327c1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdccff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4035040 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fcdd4004050 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.919+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde009c3e0 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcdd404c050 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564aebde3620 con 0x564aec0a02d0 2024-09-12T02:50:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcde009c3e0 con 0x564aec0a02d0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcdd40047e0 con 0x564aec0a02d0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 20 client.15339 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec0bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 4 client.15339 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aec011a20 con 0x564aec0bceb0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.920+0000 7fcde90f7640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec0bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.921+0000 7fcde90f7640 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec0bf2a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb51c93e3fdc4bf server_cookie=ac11189e7c8fc93 in_seq=0 out_seq=0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.921+0000 7fcdccff9640 10 client.15339 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdccff9640 1 -- 172.21.15.46:0/2180413425 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564aec011a20 con 0x564aec0bceb0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdccff9640 10 client.15339 handle_command_reply: tid=0 2024-09-12T02:50:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 client.15339 shutdown 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 msgr2=0x564aec0aa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009900 tx=0x7fcdd402aa60 comp rx=0 tx=0).stop 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 shutdown_connections 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec0bf2a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0acec0 0x564aec0aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0a02d0 0x564aec0aa320 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdccff9640 0 client.15339 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/2180413425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 >> 172.21.15.46:0/2180413425 conn(0x564aec09c060 msgr2=0x564aec0a1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 shutdown_connections 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 1 -- 172.21.15.46:0/2180413425 wait complete. 2024-09-12T02:50:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.922+0000 7fcdee1cf740 20 client.15339 trim_cache size 0 max 16384 2024-09-12T02:50:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 Processor -- start 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 -- start start 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0a02d0 0x564aec0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41af590 con 0x564aec0acec0 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ace60 con 0x564aec09f9b0 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41aacb0 con 0x564aec0a02d0 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.937+0000 7fcde98f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51298/0 (socket says 172.21.15.46:51298) 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 -- 172.21.15.46:0/4207881885 learned_addr learned my addr 172.21.15.46:0/4207881885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde90f7640 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde88f6640 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0a02d0 0x564aec0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 -- 172.21.15.46:0/4207881885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0a02d0 msgr2=0x564aec0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0a02d0 0x564aec0c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 -- 172.21.15.46:0/4207881885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 msgr2=0x564aec0abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 -- 172.21.15.46:0/4207881885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aec0bf610 con 0x564aec0acec0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcde98f8640 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011890 tx=0x7fcddc0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c7c2ca846f59cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcdceffd640 1 -- 172.21.15.46:0/4207881885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc024070 con 0x564aec0acec0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcdceffd640 1 -- 172.21.15.46:0/4207881885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcddc01e450 con 0x564aec0acec0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.938+0000 7fcdceffd640 1 -- 172.21.15.46:0/4207881885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc028660 con 0x564aec0acec0 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- 172.21.15.46:0/4207881885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 msgr2=0x564aec0c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011890 tx=0x7fcddc0097b0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- 172.21.15.46:0/4207881885 shutdown_connections 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0c50f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec0a02d0 0x564aec0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/4207881885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec09f9b0 0x564aec0abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- 172.21.15.46:0/4207881885 >> 172.21.15.46:0/4207881885 conn(0x564aec09c060 msgr2=0x564aec0ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- 172.21.15.46:0/4207881885 shutdown_connections 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- 172.21.15.46:0/4207881885 wait complete. 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 -- start start 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.939+0000 7fcdee1cf740 1 Processor -- start 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec09f9b0 0x564aec0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec0a02d0 0x564aec0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ab320 con 0x564aec0acec0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ac790 con 0x564aec0a02d0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ab9f0 con 0x564aec09f9b0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdee1cf740 10 client.0 _ll_register_callbacks cb 0x7fcdd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51304/0 (socket says 172.21.15.46:51304) 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 -- 172.21.15.46:0/445071981 learned_addr learned my addr 172.21.15.46:0/445071981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde90f7640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec09f9b0 0x564aec0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde88f6640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec0a02d0 0x564aec0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 -- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec09f9b0 msgr2=0x564aec0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec09f9b0 0x564aec0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 -- 172.21.15.46:0/445071981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec0a02d0 msgr2=0x564aec0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec0a02d0 0x564aec0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 -- 172.21.15.46:0/445071981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aec0b1240 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcde98f8640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011860 tx=0x7fcddc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcad452f34a9666 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.940+0000 7fcdccff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc036020 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fcddc01d8e0 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcddc01e450 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcddc041050 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 10 client.15345 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564aec1279a0 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcddc01e450 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcddc01b040 con 0x564aec0acec0 2024-09-12T02:50:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 20 client.15345 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec12a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 4 client.15345 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.941+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aec00bd70 con 0x564aec0bceb0 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.942+0000 7fcde90f7640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec12a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.942+0000 7fcde90f7640 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec12a0a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97e67f08dbd8c9a server_cookie=d792a8145c6a810a in_seq=0 out_seq=0 2024-09-12T02:50:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.942+0000 7fcdccff9640 10 client.15345 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdccff9640 1 -- 172.21.15.46:0/445071981 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564aec00bd70 con 0x564aec0bceb0 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdccff9640 10 client.15345 handle_command_reply: tid=0 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 client.15345 shutdown 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 msgr2=0x564aec0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011860 tx=0x7fcddc009f00 comp rx=0 tx=0).stop 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 shutdown_connections 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aec0bceb0 0x564aec12a0a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aec0acec0 0x564aec0b0c00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aec0a02d0 0x564aec0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdccff9640 0 client.15345 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.954+0000 7fcdee1cf740 1 --2- 172.21.15.46:0/445071981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aec09f9b0 0x564aec0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 >> 172.21.15.46:0/445071981 conn(0x564aec09c060 msgr2=0x564aec09b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 shutdown_connections 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdee1cf740 1 -- 172.21.15.46:0/445071981 wait complete. 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdee1cf740 20 client.15345 trim_cache size 0 max 16384 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:50:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:50:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:50:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:50:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 msgr2=0x7fcdd00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 0x7fcdd00a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009370 tx=0x7fcddc004ab0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 msgr2=0x7fcde41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e9f0 tx=0x7fcde000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 shutdown_connections 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcdd00a2ec0 0x7fcdd00a5380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde41069e0 0x7fcde419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcde4101370 0x7fcde4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 --2- 172.21.15.46:0/1079918799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde41009d0 0x7fcde41971c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.955+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 >> 172.21.15.46:0/1079918799 conn(0x7fcde40fc6f0 msgr2=0x7fcde40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.956+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 shutdown_connections 2024-09-12T02:50:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:25.956+0000 7fcdeb35b640 1 -- 172.21.15.46:0/1079918799 wait complete. 2024-09-12T02:50:27.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:27 smithi046 ceph-mon[30951]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-09-12T02:50:27.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:27 smithi145 ceph-mon[34674]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-09-12T02:50:27.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:27 smithi196 ceph-mon[34587]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-09-12T02:50:29.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:29 smithi046 ceph-mon[30951]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-12T02:50:29.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:29 smithi145 ceph-mon[34674]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-12T02:50:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:29 smithi196 ceph-mon[34587]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-12T02:50:31.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:31 smithi046 ceph-mon[30951]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:50:31.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:31 smithi145 ceph-mon[34674]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:50:31.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:31 smithi196 ceph-mon[34587]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T02:50:33.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:33 smithi046 ceph-mon[30951]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:50:33.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:33 smithi145 ceph-mon[34674]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:50:33.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:33 smithi196 ceph-mon[34587]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:50:35.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:35 smithi046 ceph-mon[30951]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T02:50:35.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:35 smithi145 ceph-mon[34674]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T02:50:35.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:35 smithi196 ceph-mon[34587]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T02:50:37.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:37 smithi046 ceph-mon[30951]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-12T02:50:37.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:37 smithi145 ceph-mon[34674]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-12T02:50:37.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:37 smithi196 ceph-mon[34587]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 -- 172.21.15.46:0/2250195649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac1069c0 msgr2=0x7fc2ac106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2250195649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac106da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009930 tx=0x7fc2a802a420 comp rx=0 tx=0).stop 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 -- 172.21.15.46:0/2250195649 shutdown_connections 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2250195649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac101350 0x7fc2ac10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2250195649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 0x7fc2ac100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2250195649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac106da0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 -- 172.21.15.46:0/2250195649 >> 172.21.15.46:0/2250195649 conn(0x7fc2ac077990 msgr2=0x7fc2ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 -- 172.21.15.46:0/2250195649 shutdown_connections 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.050+0000 7fc2b447b640 1 -- 172.21.15.46:0/2250195649 wait complete. 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 Processor -- start 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 -- start start 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 0x7fc2ac1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac1a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac110cc0 con 0x7fc2ac101350 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac110b40 con 0x7fc2ac1069c0 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b447b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac110e40 con 0x7fc2ac1009b0 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b1a16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b1a16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57380/0 (socket says 172.21.15.46:57380) 2024-09-12T02:50:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2357273338 learned_addr learned my addr 172.21.15.46:0/2357273338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.051+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 0x7fc2ac1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac1a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2357273338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 msgr2=0x7fc2ac1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 0x7fc2ac1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2357273338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac1069c0 msgr2=0x7fc2ac1a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac1a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2357273338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1a46b0 con 0x7fc2ac101350 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c920 tx=0x7fc29c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2a37fe640 1 -- 172.21.15.46:0/2357273338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c007c70 con 0x7fc2ac101350 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1a4880 con 0x7fc2ac101350 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2a37fe640 1 -- 172.21.15.46:0/2357273338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc29c007e10 con 0x7fc2ac101350 2024-09-12T02:50:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2a37fe640 1 -- 172.21.15.46:0/2357273338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c018820 con 0x7fc2ac101350 2024-09-12T02:50:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.052+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a4da0 con 0x7fc2ac101350 2024-09-12T02:50:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2a37fe640 1 -- 172.21.15.46:0/2357273338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fc29c0189c0 con 0x7fc2ac101350 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 Processor -- start 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 -- start start 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e00ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e0b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e0e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a777d652f0 con 0x55a777e0bcb0 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a777d0d270 con 0x55a777e00af0 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.055+0000 7fc2b721c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a777d612b0 con 0x55a777e01410 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2217640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e0b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e00ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b1a16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e0e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2217640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e0b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:45006/0 (socket says 172.21.15.46:45006) 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2217640 1 -- 172.21.15.46:0/1133710828 learned_addr learned my addr 172.21.15.46:0/1133710828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2a37fe640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 0x7fc2980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 -- 172.21.15.46:0/1133710828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 msgr2=0x55a777e0b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e0b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 -- 172.21.15.46:0/1133710828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 msgr2=0x55a777e0e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e0e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 -- 172.21.15.46:0/1133710828 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a777d0c340 con 0x55a777e00af0 2024-09-12T02:50:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2a37fe640 1 -- 172.21.15.46:0/2357273338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc29c0cfb50 con 0x7fc2ac101350 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 0x7fc2980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.056+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e00ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009ca0 tx=0x7fc2a400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58aa8dc667daf06d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2a0bfe640 1 -- 172.21.15.46:0/1133710828 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003c50 con 0x55a777e00af0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2a0bfe640 1 -- 172.21.15.46:0/1133710828 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2a401c920 con 0x55a777e00af0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2a0bfe640 1 -- 172.21.15.46:0/1133710828 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40117a0 con 0x55a777e00af0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 0x7fc2980a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2a401b000 tx=0x7fc2a401e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 -- 172.21.15.46:0/1133710828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 msgr2=0x55a777e00ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e00ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009ca0 tx=0x7fc2a400f730 comp rx=0 tx=0).stop 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 -- 172.21.15.46:0/1133710828 shutdown_connections 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e0e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e0b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 --2- 172.21.15.46:0/1133710828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e00ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.057+0000 7fc2b721c740 1 -- 172.21.15.46:0/1133710828 >> 172.21.15.46:0/1133710828 conn(0x55a777dfd060 msgr2=0x55a777dfd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- 172.21.15.46:0/1133710828 shutdown_connections 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- 172.21.15.46:0/1133710828 wait complete. 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- start start 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 Processor -- start 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a777e12c90 con 0x55a777e01410 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a777cee290 con 0x55a777e00af0 2024-09-12T02:50:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a777e12e10 con 0x55a777e0bcb0 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b721c740 10 client.0 _ll_register_callbacks cb 0x55a777d1f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b1a16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b2217640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b1a16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57398/0 (socket says 172.21.15.46:57398) 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2052408662 learned_addr learned my addr 172.21.15.46:0/2052408662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.058+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 msgr2=0x55a777e096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2052408662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 msgr2=0x55a777e08c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e08c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 -- 172.21.15.46:0/2052408662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a777a78210 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007590 tx=0x7fc29c097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74d1f54f3575d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c018cb0 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc29c0a4580 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a401c920 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc29c0cd340 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b721c740 10 client.15357 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a777b44620 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.059+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2a401c920 con 0x55a777e01410 2024-09-12T02:50:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc29c0ba2e0 con 0x55a777e01410 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 20 client.15357 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e20120 0x55a777e22510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 4 client.15357 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a777d72a20 con 0x55a777e20120 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e20120 0x55a777e22510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e20120 0x55a777e22510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5e20469dba6abf server_cookie=a15f74737e31769b in_seq=0 out_seq=0 2024-09-12T02:50:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.060+0000 7fc295ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc295ffb640 1 -- 172.21.15.46:0/2052408662 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a777d72a20 con 0x55a777e20120 2024-09-12T02:50:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc295ffb640 10 client.15357 handle_command_reply: tid=0 2024-09-12T02:50:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 client.15357 shutdown 2024-09-12T02:50:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 msgr2=0x55a777e09190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007590 tx=0x7fc29c097f00 comp rx=0 tx=0).stop 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 shutdown_connections 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e20120 0x55a777e22510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e01410 0x55a777e09190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc295ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2052408662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 >> 172.21.15.46:0/2052408662 conn(0x55a777dfd060 msgr2=0x55a777e02dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 shutdown_connections 2024-09-12T02:50:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.061+0000 7fc2b721c740 1 -- 172.21.15.46:0/2052408662 wait complete. 2024-09-12T02:50:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.062+0000 7fc2b721c740 20 client.15357 trim_cache size 0 max 16384 2024-09-12T02:50:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 Processor -- start 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 -- start start 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e01410 0x55a777e27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1af3c0 con 0x55a777e00af0 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1acc90 con 0x55a777e01410 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b721c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1aaae0 con 0x55a777e0bcb0 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b2217640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b2217640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57414/0 (socket says 172.21.15.46:57414) 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b2217640 1 -- 172.21.15.46:0/4278447170 learned_addr learned my addr 172.21.15.46:0/4278447170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.076+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e01410 0x55a777e27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 -- 172.21.15.46:0/4278447170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 msgr2=0x55a777e282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 -- 172.21.15.46:0/4278447170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e01410 msgr2=0x55a777e27db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e01410 0x55a777e27db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 -- 172.21.15.46:0/4278447170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a777e01240 con 0x55a777e00af0 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc2b2217640 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a940 tx=0x7fc2a8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfee5ae2736aff09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc297fff640 1 -- 172.21.15.46:0/4278447170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8035040 con 0x55a777e00af0 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc297fff640 1 -- 172.21.15.46:0/4278447170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2a804a070 con 0x55a777e00af0 2024-09-12T02:50:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.077+0000 7fc297fff640 1 -- 172.21.15.46:0/4278447170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80035f0 con 0x55a777e00af0 2024-09-12T02:50:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- 172.21.15.46:0/4278447170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 msgr2=0x55a777e09320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a940 tx=0x7fc2a8005a20 comp rx=0 tx=0).stop 2024-09-12T02:50:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- 172.21.15.46:0/4278447170 shutdown_connections 2024-09-12T02:50:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e0bcb0 0x55a777e282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e01410 0x55a777e27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 --2- 172.21.15.46:0/4278447170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e00af0 0x55a777e09320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- 172.21.15.46:0/4278447170 >> 172.21.15.46:0/4278447170 conn(0x55a777dfd060 msgr2=0x55a777e08c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- 172.21.15.46:0/4278447170 shutdown_connections 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- 172.21.15.46:0/4278447170 wait complete. 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 -- start start 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.078+0000 7fc2b721c740 1 Processor -- start 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e13d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ab150 con 0x55a777e0bcb0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ac5c0 con 0x55a777e00af0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ab820 con 0x55a777e01410 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b721c740 10 client.0 _ll_register_callbacks cb 0x7fc29c00a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57430/0 (socket says 172.21.15.46:57430) 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 -- 172.21.15.46:0/2920270273 learned_addr learned my addr 172.21.15.46:0/2920270273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e13d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b1a16640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 -- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 msgr2=0x55a777e142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 -- 172.21.15.46:0/2920270273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 msgr2=0x55a777e13d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e13d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 -- 172.21.15.46:0/2920270273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a777e89d80 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc2b2a18640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c078b8d36d0a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.079+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40116d0 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc2a4003970 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a804a070 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4003b70 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 10 client.15363 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a777e8aff0 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2a804a070 con 0x55a777e0bcb0 2024-09-12T02:50:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2a401c470 con 0x55a777e0bcb0 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 20 client.15363 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e8b5b0 0x55a777e8d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 4 client.15363 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a777d6cd70 con 0x55a777e8b5b0 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.080+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e8b5b0 0x55a777e8d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.081+0000 7fc2b2217640 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e8b5b0 0x55a777e8d9a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0cb82b87ee3dc5 server_cookie=cd9374ed6178b883 in_seq=0 out_seq=0 2024-09-12T02:50:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.081+0000 7fc295ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.084+0000 7fc295ffb640 1 -- 172.21.15.46:0/2920270273 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a777d6cd70 con 0x55a777e8b5b0 2024-09-12T02:50:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc295ffb640 10 client.15363 handle_command_reply: tid=0 2024-09-12T02:50:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 client.15363 shutdown 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 msgr2=0x55a777e147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4003fa0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 shutdown_connections 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a777e8b5b0 0x55a777e8d9a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a777e0bcb0 0x55a777e147e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a777e01410 0x55a777e142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc295ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 --2- 172.21.15.46:0/2920270273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a777e00af0 0x55a777e13d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 >> 172.21.15.46:0/2920270273 conn(0x55a777dfd060 msgr2=0x55a777e09430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 shutdown_connections 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 1 -- 172.21.15.46:0/2920270273 wait complete. 2024-09-12T02:50:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b721c740 20 client.15363 trim_cache size 0 max 16384 2024-09-12T02:50:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:50:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:50:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:50:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 msgr2=0x7fc2980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 0x7fc2980a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2a401b000 tx=0x7fc2a401e000 comp rx=0 tx=0).stop 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 msgr2=0x7fc2ac199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c920 tx=0x7fc29c00cde0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 shutdown_connections 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc2980a2e40 0x7fc2980a5300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2ac1069c0 0x7fc2ac1a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2ac101350 0x7fc2ac199630 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.085+0000 7fc2b447b640 1 --2- 172.21.15.46:0/2357273338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2ac1009b0 0x7fc2ac1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.086+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 >> 172.21.15.46:0/2357273338 conn(0x7fc2ac077990 msgr2=0x7fc2ac1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.086+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 shutdown_connections 2024-09-12T02:50:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:39.086+0000 7fc2b447b640 1 -- 172.21.15.46:0/2357273338 wait complete. 2024-09-12T02:50:39.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:39 smithi046 ceph-mon[30951]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-12T02:50:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:39 smithi145 ceph-mon[34674]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-12T02:50:39.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:39 smithi196 ceph-mon[34587]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-12T02:50:41.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:41 smithi046 ceph-mon[30951]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:41 smithi145 ceph-mon[34674]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:41.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:41 smithi196 ceph-mon[34587]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:43.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:43 smithi046 ceph-mon[30951]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:43 smithi145 ceph-mon[34674]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:43 smithi196 ceph-mon[34587]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:45.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:45 smithi046 ceph-mon[30951]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:45 smithi145 ceph-mon[34674]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:45 smithi196 ceph-mon[34587]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:47.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:47 smithi046 ceph-mon[30951]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:47 smithi145 ceph-mon[34674]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:47 smithi196 ceph-mon[34587]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:49.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:49 smithi046 ceph-mon[30951]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:49 smithi145 ceph-mon[34674]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:49.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:49 smithi196 ceph-mon[34587]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:51.773 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:51 smithi196 ceph-mon[34587]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:51.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:51 smithi046 ceph-mon[30951]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:51 smithi145 ceph-mon[34674]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 -- 172.21.15.46:0/3779571966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 msgr2=0x7f46cc103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 --2- 172.21.15.46:0/3779571966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc103e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009900 tx=0x7f46bc02a410 comp rx=0 tx=0).stop 2024-09-12T02:50:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 -- 172.21.15.46:0/3779571966 shutdown_connections 2024-09-12T02:50:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 --2- 172.21.15.46:0/3779571966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc1008e0 0x7f46cc10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 --2- 172.21.15.46:0/3779571966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc0fff60 0x7f46cc1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 --2- 172.21.15.46:0/3779571966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc103e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 -- 172.21.15.46:0/3779571966 >> 172.21.15.46:0/3779571966 conn(0x7f46cc0770f0 msgr2=0x7f46cc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 -- 172.21.15.46:0/3779571966 shutdown_connections 2024-09-12T02:50:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.222+0000 7f46d172a640 1 -- 172.21.15.46:0/3779571966 wait complete. 2024-09-12T02:50:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.223+0000 7f46d172a640 1 Processor -- start 2024-09-12T02:50:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.223+0000 7f46d172a640 1 -- start start 2024-09-12T02:50:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.223+0000 7f46d172a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 0x7f46cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.223+0000 7f46d172a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.223+0000 7f46d172a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46d172a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc111f10 con 0x7f46cc1008e0 2024-09-12T02:50:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46d172a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc111d90 con 0x7f46cc0fff60 2024-09-12T02:50:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41402/0 (socket says 172.21.15.46:41402) 2024-09-12T02:50:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46ca7fc640 1 -- 172.21.15.46:0/83880748 learned_addr learned my addr 172.21.15.46:0/83880748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46d172a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc112090 con 0x7f46cc103aa0 2024-09-12T02:50:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 0x7f46cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 -- 172.21.15.46:0/83880748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 msgr2=0x7f46cc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 -- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 msgr2=0x7f46cc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46caffd640 1 -- 172.21.15.46:0/83880748 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19d3f0 con 0x7f46cc0fff60 2024-09-12T02:50:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.224+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46caffd640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 0x7f46cc198400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46bc031440 tx=0x7f46bc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46b3fff640 1 -- 172.21.15.46:0/83880748 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc03f070 con 0x7f46cc0fff60 2024-09-12T02:50:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46b3fff640 1 -- 172.21.15.46:0/83880748 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46bc003aa0 con 0x7f46cc0fff60 2024-09-12T02:50:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc19d680 con 0x7f46cc0fff60 2024-09-12T02:50:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a4f70 con 0x7f46cc0fff60 2024-09-12T02:50:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.225+0000 7f46b3fff640 1 -- 172.21.15.46:0/83880748 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc035070 con 0x7f46cc0fff60 2024-09-12T02:50:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.228+0000 7f46d44c3740 1 Processor -- start 2024-09-12T02:50:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.228+0000 7f46b3fff640 1 -- 172.21.15.46:0/83880748 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f46bc003c50 con 0x7f46cc0fff60 2024-09-12T02:50:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.228+0000 7f46d44c3740 1 -- start start 2024-09-12T02:50:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbe85112f0 con 0x55fbe85acaf0 2024-09-12T02:50:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fbe84b9270 con 0x55fbe85ad410 2024-09-12T02:50:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46d44c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fbe850d2b0 con 0x55fbe85b7cb0 2024-09-12T02:50:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46772/0 (socket says 172.21.15.46:46772) 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41406/0 (socket says 172.21.15.46:41406) 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 -- 172.21.15.46:0/2442985037 learned_addr learned my addr 172.21.15.46:0/2442985037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 -- 172.21.15.46:0/2442985037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 msgr2=0x55fbe85b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 -- 172.21.15.46:0/2442985037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 msgr2=0x55fbe85aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46ca7fc640 1 -- 172.21.15.46:0/2442985037 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe84b8340 con 0x55fbe85b7cb0 2024-09-12T02:50:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.229+0000 7f46b3fff640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 0x7f46b40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46c00115f0 tx=0x7f46c0011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c8c33f27f46474 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 0x7f46b40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46b3fff640 1 -- 172.21.15.46:0/83880748 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f46bc0efca0 con 0x7f46cc0fff60 2024-09-12T02:50:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46b17fa640 1 -- 172.21.15.46:0/2442985037 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c001e070 con 0x55fbe85b7cb0 2024-09-12T02:50:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46b17fa640 1 -- 172.21.15.46:0/2442985037 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c000fb10 con 0x55fbe85b7cb0 2024-09-12T02:50:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.230+0000 7f46b17fa640 1 -- 172.21.15.46:0/2442985037 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0019bc0 con 0x55fbe85b7cb0 2024-09-12T02:50:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 -- 172.21.15.46:0/2442985037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 msgr2=0x55fbe85ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46c00115f0 tx=0x7f46c0011ab0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 0x7f46b40a5400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009fd0 tx=0x7f46c4009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 -- 172.21.15.46:0/2442985037 shutdown_connections 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85b7cb0 0x55fbe85ba0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 --2- 172.21.15.46:0/2442985037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85acaf0 0x55fbe85aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 -- 172.21.15.46:0/2442985037 >> 172.21.15.46:0/2442985037 conn(0x55fbe85a9060 msgr2=0x55fbe85a9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 -- 172.21.15.46:0/2442985037 shutdown_connections 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.231+0000 7f46d44c3740 1 -- 172.21.15.46:0/2442985037 wait complete. 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 -- start start 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 Processor -- start 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbe85ba880 con 0x55fbe85b7cb0 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fbe849a290 con 0x55fbe85ad410 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fbe85baa00 con 0x55fbe85acaf0 2024-09-12T02:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46caffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46caffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46776/0 (socket says 172.21.15.46:46776) 2024-09-12T02:50:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46caffd640 1 -- 172.21.15.46:0/3161923467 learned_addr learned my addr 172.21.15.46:0/3161923467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46d44c3740 10 client.0 _ll_register_callbacks cb 0x55fbe84cb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.232+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 -- 172.21.15.46:0/3161923467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 msgr2=0x55fbe85b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 -- 172.21.15.46:0/3161923467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85b7cb0 msgr2=0x55fbe85b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 -- 172.21.15.46:0/3161923467 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe8224210 con 0x55fbe85acaf0 2024-09-12T02:50:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46caffd640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46bc02a930 tx=0x7f46bc0bb7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0022b51dd00b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T02:50:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0bbc90 con 0x55fbe85acaf0 2024-09-12T02:50:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c000fb10 con 0x55fbe85acaf0 2024-09-12T02:50:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f46bc03b460 con 0x55fbe85acaf0 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46bc03b660 con 0x55fbe85acaf0 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46d44c3740 10 client.25024 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbe82f0620 con 0x55fbe85acaf0 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.233+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c000fb10 con 0x55fbe85acaf0 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f46bc0d4070 con 0x55fbe85acaf0 2024-09-12T02:50:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 10 client.25024 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 10 client.25024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 20 client.25024 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe85c7e80 0x55fbe85ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 4 client.25024 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:50:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbe851ea20 con 0x55fbe85c7e80 2024-09-12T02:50:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe85c7e80 0x55fbe85ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe85c7e80 0x55fbe85ca300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c87ff2156df49b2 server_cookie=876b6bee9d4a94e4 in_seq=0 out_seq=0 2024-09-12T02:50:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.234+0000 7f46aeff5640 10 client.25024 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.235+0000 7f46aeff5640 1 -- 172.21.15.46:0/3161923467 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbe851ea20 con 0x55fbe85c7e80 2024-09-12T02:50:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.235+0000 7f46aeff5640 10 client.25024 handle_command_reply: tid=0 2024-09-12T02:50:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.235+0000 7f46d44c3740 1 client.25024 shutdown 2024-09-12T02:50:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 msgr2=0x55fbe85b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46bc02a930 tx=0x7f46bc0bb7d0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 shutdown_connections 2024-09-12T02:50:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe85c7e80 0x55fbe85ca300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85ad410 0x55fbe85b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 --2- 172.21.15.46:0/3161923467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b4c30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 >> 172.21.15.46:0/3161923467 conn(0x55fbe85a9060 msgr2=0x55fbe85aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46aeff5640 0 client.25024 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 shutdown_connections 2024-09-12T02:50:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 1 -- 172.21.15.46:0/3161923467 wait complete. 2024-09-12T02:50:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.236+0000 7f46d44c3740 20 client.25024 trim_cache size 0 max 16384 2024-09-12T02:50:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 Processor -- start 2024-09-12T02:50:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 -- start start 2024-09-12T02:50:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8012650 con 0x55fbe85ad410 2024-09-12T02:50:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800ff20 con 0x55fbe85b7cb0 2024-09-12T02:50:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.265+0000 7f46d44c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800dd70 con 0x55fbe85acaf0 2024-09-12T02:50:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41432/0 (socket says 172.21.15.46:41432) 2024-09-12T02:50:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 -- 172.21.15.46:0/1512884370 learned_addr learned my addr 172.21.15.46:0/1512884370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46caffd640 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 -- 172.21.15.46:0/1512884370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 msgr2=0x55fbe85b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 -- 172.21.15.46:0/1512884370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 msgr2=0x55fbe85cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 -- 172.21.15.46:0/1512884370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe85ba760 con 0x55fbe85ad410 2024-09-12T02:50:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009230 tx=0x7f46c0019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d79d6a91e129273 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46b0ff9640 1 -- 172.21.15.46:0/1512884370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c001e040 con 0x55fbe85ad410 2024-09-12T02:50:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46b0ff9640 1 -- 172.21.15.46:0/1512884370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c000fb10 con 0x55fbe85ad410 2024-09-12T02:50:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.266+0000 7f46b0ff9640 1 -- 172.21.15.46:0/1512884370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00197b0 con 0x55fbe85ad410 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 -- 172.21.15.46:0/1512884370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 msgr2=0x55fbe85b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009230 tx=0x7f46c0019410 comp rx=0 tx=0).stop 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 -- 172.21.15.46:0/1512884370 shutdown_connections 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b5720 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 --2- 172.21.15.46:0/1512884370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 -- 172.21.15.46:0/1512884370 >> 172.21.15.46:0/1512884370 conn(0x55fbe85a9060 msgr2=0x55fbe85b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 -- 172.21.15.46:0/1512884370 shutdown_connections 2024-09-12T02:50:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.267+0000 7f46d44c3740 1 -- 172.21.15.46:0/1512884370 wait complete. 2024-09-12T02:50:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 -- start start 2024-09-12T02:50:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 Processor -- start 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800e3e0 con 0x55fbe85ad410 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800f850 con 0x55fbe85b7cb0 2024-09-12T02:50:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800eab0 con 0x55fbe85acaf0 2024-09-12T02:50:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46d44c3740 10 client.0 _ll_register_callbacks cb 0x7f46bc0ece00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:50:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46ca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46caffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46caffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46800/0 (socket says 172.21.15.46:46800) 2024-09-12T02:50:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46caffd640 1 -- 172.21.15.46:0/604788604 learned_addr learned my addr 172.21.15.46:0/604788604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.268+0000 7f46cb7fe640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 -- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 msgr2=0x55fbe85c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 -- 172.21.15.46:0/604788604 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 msgr2=0x55fbe85b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 -- 172.21.15.46:0/604788604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe86319e0 con 0x55fbe85ad410 2024-09-12T02:50:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46caffd640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46ca7fc640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b1ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55fbe84b45a0 tx=0x7f46c0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec796ca27db4532c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0034070 con 0x55fbe85ad410 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f46c000f870 con 0x55fbe85ad410 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c000fb10 con 0x55fbe85ad410 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46c001e040 con 0x55fbe85ad410 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46d44c3740 10 client.15381 fetch_fsmap learned FSMap version 25 2024-09-12T02:50:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.269+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbe8632b30 con 0x55fbe85ad410 2024-09-12T02:50:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c00054d0 con 0x55fbe85ad410 2024-09-12T02:50:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f46c0019cb0 con 0x55fbe85ad410 2024-09-12T02:50:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:50:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:50:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 20 client.15381 populate_metadata read hostname 'smithi046' 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe8632fa0 0x55fbe8635390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 4 client.15381 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f46c000fb10 con 0x55fbe8632fa0 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.270+0000 7f46caffd640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe8632fa0 0x55fbe8635390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.271+0000 7f46caffd640 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe8632fa0 0x55fbe8635390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b743f86841c610e2 server_cookie=6e5fb8a2e329c9e in_seq=0 out_seq=0 2024-09-12T02:50:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.271+0000 7f46aeff5640 10 client.15381 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46aeff5640 1 -- 172.21.15.46:0/604788604 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f46c000fb10 con 0x55fbe8632fa0 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46aeff5640 10 client.15381 handle_command_reply: tid=0 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 client.15381 shutdown 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 msgr2=0x55fbe85b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b1ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55fbe84b45a0 tx=0x7f46c0009530 comp rx=0 tx=0).stop 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 shutdown_connections 2024-09-12T02:50:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fbe8632fa0 0x55fbe8635390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fbe85b7cb0 0x55fbe85b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbe85ad410 0x55fbe85b1ad0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46aeff5640 0 client.15381 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.274+0000 7f46d44c3740 1 --2- 172.21.15.46:0/604788604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbe85acaf0 0x55fbe85c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 >> 172.21.15.46:0/604788604 conn(0x55fbe85a9060 msgr2=0x55fbe85ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 shutdown_connections 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d44c3740 1 -- 172.21.15.46:0/604788604 wait complete. 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d44c3740 20 client.15381 trim_cache size 0 max 16384 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:50:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:50:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:50:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 msgr2=0x7f46b40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 0x7f46b40a5400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009fd0 tx=0x7f46c4009500 comp rx=0 tx=0).stop 2024-09-12T02:50:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 msgr2=0x7f46cc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 0x7f46cc198400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46bc031440 tx=0x7f46bc004040 comp rx=0 tx=0).stop 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 shutdown_connections 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f46b40a2f40 0x7f46b40a5400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46cc103aa0 0x7f46cc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46cc1008e0 0x7f46cc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 --2- 172.21.15.46:0/83880748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46cc0fff60 0x7f46cc198400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.275+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 >> 172.21.15.46:0/83880748 conn(0x7f46cc0770f0 msgr2=0x7f46cc105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.276+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 shutdown_connections 2024-09-12T02:50:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:50:52.276+0000 7f46d172a640 1 -- 172.21.15.46:0/83880748 wait complete. 2024-09-12T02:50:53.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:53 smithi046 ceph-mon[30951]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:53.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:53 smithi145 ceph-mon[34674]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:53 smithi196 ceph-mon[34587]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:55 smithi046 ceph-mon[30951]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:55.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:55 smithi145 ceph-mon[34674]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:55.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:55 smithi196 ceph-mon[34587]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T02:50:57.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:57 smithi046 ceph-mon[30951]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:57 smithi145 ceph-mon[34674]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:57 smithi196 ceph-mon[34587]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T02:50:59.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:50:59 smithi046 ceph-mon[30951]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:50:59 smithi145 ceph-mon[34674]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:50:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:50:59 smithi196 ceph-mon[34587]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:51:01.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:51:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:51:00.938+0000 7fe1febd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:51:01.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:51:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:51:00.938+0000 7fe1febd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:51:01.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:51:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:51:00.938+0000 7fe1febd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:51:01.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:51:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:51:00.938+0000 7fe1febd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:51:01.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 02:51:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T02:51:00.938+0000 7fe1febd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:51:01.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:01 smithi046 ceph-mon[30951]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:51:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:01 smithi145 ceph-mon[34674]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:51:01.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:01 smithi196 ceph-mon[34587]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T02:51:03.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:03 smithi046 ceph-mon[30951]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T02:51:03.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:03 smithi046 ceph-mon[30951]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-09-12T02:51:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:03 smithi145 ceph-mon[34674]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T02:51:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:03 smithi145 ceph-mon[34674]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-09-12T02:51:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:03 smithi196 ceph-mon[34587]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T02:51:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:03 smithi196 ceph-mon[34587]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-09-12T02:51:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 -- 172.21.15.46:0/1725999543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79941069c0 msgr2=0x7f7994106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 --2- 172.21.15.46:0/1725999543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79941069c0 0x7f7994106da0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7990009980 tx=0x7f799002a480 comp rx=0 tx=0).stop 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 -- 172.21.15.46:0/1725999543 shutdown_connections 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 --2- 172.21.15.46:0/1725999543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7994101350 0x7f799410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 --2- 172.21.15.46:0/1725999543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f7994100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 --2- 172.21.15.46:0/1725999543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79941069c0 0x7f7994106da0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 -- 172.21.15.46:0/1725999543 >> 172.21.15.46:0/1725999543 conn(0x7f7994077990 msgr2=0x7f79940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 -- 172.21.15.46:0/1725999543 shutdown_connections 2024-09-12T02:51:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.409+0000 7f799b874640 1 -- 172.21.15.46:0/1725999543 wait complete. 2024-09-12T02:51:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.410+0000 7f799b874640 1 Processor -- start 2024-09-12T02:51:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.410+0000 7f799b874640 1 -- start start 2024-09-12T02:51:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.410+0000 7f799b874640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f79941982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.410+0000 7f799b874640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f799b874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79941069c0 0x7f799419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f799b874640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7994110d00 con 0x7f7994101350 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7998e0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f799b874640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7994110b80 con 0x7f79941009b0 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7998e0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55336/0 (socket says 172.21.15.46:55336) 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f79941982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7998e0f640 1 -- 172.21.15.46:0/2236367535 learned_addr learned my addr 172.21.15.46:0/2236367535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7994110e80 con 0x7f79941069c0 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999e11640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79941069c0 0x7f799419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 -- 172.21.15.46:0/2236367535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79941069c0 msgr2=0x7f799419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79941069c0 0x7f799419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 -- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 msgr2=0x7f7994198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7999610640 1 -- 172.21.15.46:0/2236367535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419d2e0 con 0x7f79941009b0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.411+0000 7f7998e0f640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f7999610640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f79941982f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f799002a9a0 tx=0x7f7990004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f798a7fc640 1 -- 172.21.15.46:0/2236367535 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799003f070 con 0x7f79941009b0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f798a7fc640 1 -- 172.21.15.46:0/2236367535 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7990005430 con 0x7f79941009b0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419d570 con 0x7f79941009b0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f798a7fc640 1 -- 172.21.15.46:0/2236367535 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990035030 con 0x7f79941009b0 2024-09-12T02:51:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.412+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799419d9e0 con 0x7f79941009b0 2024-09-12T02:51:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.416+0000 7f798a7fc640 1 -- 172.21.15.46:0/2236367535 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f7990042070 con 0x7f79941009b0 2024-09-12T02:51:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.416+0000 7f799e617740 1 Processor -- start 2024-09-12T02:51:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.416+0000 7f798a7fc640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 0x7f79800a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.416+0000 7f798a7fc640 1 -- 172.21.15.46:0/2236367535 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f79900efd50 con 0x7f79941009b0 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 -- start start 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999e11640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 0x7f79800a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca87af0 0x5619cca87ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 0x5619cca92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5619cc9ec2f0 con 0x5619cca92cb0 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5619cc994270 con 0x5619cca88410 2024-09-12T02:51:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f799e617740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619cc9e82b0 con 0x5619cca87af0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55344/0 (socket says 172.21.15.46:55344) 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999e11640 1 -- 172.21.15.46:0/2826553639 learned_addr learned my addr 172.21.15.46:0/2826553639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7998e0f640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 0x5619cca92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999610640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca87af0 0x5619cca87ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.417+0000 7f7999e11640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 0x7f79800a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f798c005fd0 tx=0x7f798c011810 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 -- 172.21.15.46:0/2826553639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca87af0 msgr2=0x5619cca87ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca87af0 0x5619cca87ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 -- 172.21.15.46:0/2826553639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 msgr2=0x5619cca92770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 0x5619cca92770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 -- 172.21.15.46:0/2826553639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cc993340 con 0x5619cca92cb0 2024-09-12T02:51:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7998e0f640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 0x5619cca92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f7999e11640 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f798c003cb0 tx=0x7f798c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fe719c0fb2107e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f797f7fe640 1 -- 172.21.15.46:0/2826553639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c017040 con 0x5619cca92cb0 2024-09-12T02:51:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f797f7fe640 1 -- 172.21.15.46:0/2826553639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f798c019070 con 0x5619cca92cb0 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.418+0000 7f797f7fe640 1 -- 172.21.15.46:0/2826553639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00f860 con 0x5619cca92cb0 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 -- 172.21.15.46:0/2826553639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 msgr2=0x5619cca950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f798c003cb0 tx=0x7f798c01c000 comp rx=0 tx=0).stop 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 -- 172.21.15.46:0/2826553639 shutdown_connections 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619cca950a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca88410 0x5619cca92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 --2- 172.21.15.46:0/2826553639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca87af0 0x5619cca87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 -- 172.21.15.46:0/2826553639 >> 172.21.15.46:0/2826553639 conn(0x5619cca84060 msgr2=0x5619cca84440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 -- 172.21.15.46:0/2826553639 shutdown_connections 2024-09-12T02:51:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.419+0000 7f799e617740 1 -- 172.21.15.46:0/2826553639 wait complete. 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f799e617740 1 -- start start 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f799e617740 1 Processor -- start 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f799e617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f799e617740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f7999610640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.420+0000 7f7999610640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55358/0 (socket says 172.21.15.46:55358) 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 -- 172.21.15.46:0/1640803791 learned_addr learned my addr 172.21.15.46:0/1640803791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 0x5619cca90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5619cca95820 con 0x5619cca87af0 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5619cc975290 con 0x5619cca92cb0 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619cca959a0 con 0x5619cca88410 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 -- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 msgr2=0x5619cca8c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999e11640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 0x5619cca90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca8c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 -- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 msgr2=0x5619cca90750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 0x5619cca90750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cc6ff210 con 0x5619cca87af0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 10 client.0 _ll_register_callbacks cb 0x5619cc9a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999e11640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 0x5619cca90750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c019070 con 0x5619cca87af0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f7999610640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f79900b82a0 tx=0x7f79900bbf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6c399722c3fbc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f797cff9640 10 client.15399 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f797cff9640 1 -- 172.21.15.46:0/1640803791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990049040 con 0x5619cca87af0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f797cff9640 1 -- 172.21.15.46:0/1640803791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f7990003b30 con 0x5619cca87af0 2024-09-12T02:51:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f797cff9640 1 -- 172.21.15.46:0/1640803791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79900c4570 con 0x5619cca87af0 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.421+0000 7f799e617740 10 client.15399 fetch_fsmap learned FSMap version 25 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5619cc7cb620 con 0x5619cca87af0 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f797cff9640 1 -- 172.21.15.46:0/1640803791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f799004c060 con 0x5619cca87af0 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 10 client.15399 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 20 client.15399 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccaab200 0x5619ccaad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 4 client.15399 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619cc9f9a20 con 0x5619ccaab200 2024-09-12T02:51:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.422+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccaab200 0x5619ccaad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.423+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccaab200 0x5619ccaad650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712d01be3ee8b8e7 server_cookie=d816df9f2b0eb0c0 in_seq=0 out_seq=0 2024-09-12T02:51:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.423+0000 7f797cff9640 10 client.15399 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.423+0000 7f797cff9640 1 -- 172.21.15.46:0/1640803791 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619cc9f9a20 con 0x5619ccaab200 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.423+0000 7f797cff9640 10 client.15399 handle_command_reply: tid=0 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.423+0000 7f799e617740 1 client.15399 shutdown 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 msgr2=0x5619cca8bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f79900b82a0 tx=0x7f79900bbf00 comp rx=0 tx=0).stop 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 shutdown_connections 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccaab200 0x5619ccaad650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca92cb0 0x5619cca90750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 --2- 172.21.15.46:0/1640803791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca87af0 0x5619cca8bec0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f797cff9640 0 client.15399 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 >> 172.21.15.46:0/1640803791 conn(0x5619cca84060 msgr2=0x5619cca89d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 shutdown_connections 2024-09-12T02:51:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 1 -- 172.21.15.46:0/1640803791 wait complete. 2024-09-12T02:51:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.424+0000 7f799e617740 20 client.15399 trim_cache size 0 max 16384 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.438+0000 7f799e617740 1 Processor -- start 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.438+0000 7f799e617740 1 -- start start 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619cca90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca96520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79941af420 con 0x5619cca92cb0 2024-09-12T02:51:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79941accf0 con 0x5619cca87af0 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f799e617740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79941aab40 con 0x5619cca88410 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55368/0 (socket says 172.21.15.46:55368) 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 -- 172.21.15.46:0/1339503724 learned_addr learned my addr 172.21.15.46:0/1339503724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca96520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999610640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619cca90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 -- 172.21.15.46:0/1339503724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 msgr2=0x5619cca96520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca96520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 -- 172.21.15.46:0/1339503724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 msgr2=0x5619cca90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619cca90740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 -- 172.21.15.46:0/1339503724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cca95410 con 0x5619cca92cb0 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca96520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:51:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.439+0000 7f7999e11640 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f798c012fb0 tx=0x7f798c012fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f430fe8fac2000 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.440+0000 7f797effd640 1 -- 172.21.15.46:0/1339503724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c017040 con 0x5619cca92cb0 2024-09-12T02:51:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.440+0000 7f797effd640 1 -- 172.21.15.46:0/1339503724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f798c005050 con 0x5619cca92cb0 2024-09-12T02:51:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.440+0000 7f797effd640 1 -- 172.21.15.46:0/1339503724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c01e6a0 con 0x5619cca92cb0 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- 172.21.15.46:0/1339503724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 msgr2=0x5619ccab2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f798c012fb0 tx=0x7f798c012fe0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- 172.21.15.46:0/1339503724 shutdown_connections 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca92cb0 0x5619ccab2f60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca88410 0x5619cca96520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 --2- 172.21.15.46:0/1339503724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619cca90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- 172.21.15.46:0/1339503724 >> 172.21.15.46:0/1339503724 conn(0x5619cca84060 msgr2=0x5619cca89d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- 172.21.15.46:0/1339503724 shutdown_connections 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- 172.21.15.46:0/1339503724 wait complete. 2024-09-12T02:51:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 -- start start 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.441+0000 7f799e617740 1 Processor -- start 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619ccaa72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca92cb0 0x5619ccaa4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79941ab1b0 con 0x5619cca88410 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79941ac620 con 0x5619cca87af0 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79941ab880 con 0x5619cca92cb0 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f799e617740 10 client.0 _ll_register_callbacks cb 0x7f79900098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55374/0 (socket says 172.21.15.46:55374) 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 -- 172.21.15.46:0/1476719066 learned_addr learned my addr 172.21.15.46:0/1476719066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7999610640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619ccaa72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7999e11640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca92cb0 0x5619ccaa4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 -- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca92cb0 msgr2=0x5619ccaa4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca92cb0 0x5619ccaa4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 -- 172.21.15.46:0/1476719066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 msgr2=0x5619ccaa72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619ccaa72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7998e0f640 1 -- 172.21.15.46:0/1476719066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ccaa47e0 con 0x5619cca88410 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.442+0000 7f7999610640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619ccaa72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f7998e0f640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5619cca84ec0 tx=0x7f7984010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143147068d9731be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f797cff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798400fca0 con 0x5619cca88410 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f798400fe40 con 0x5619cca88410 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c005050 con 0x5619cca88410 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7984004020 con 0x5619cca88410 2024-09-12T02:51:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f799e617740 10 client.15405 fetch_fsmap learned FSMap version 25 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619ccb167f0 con 0x5619cca88410 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.443+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f798c005050 con 0x5619cca88410 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f798401e410 con 0x5619cca88410 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 20 client.15405 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccb16f40 0x5619ccb19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 4 client.15405 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619cc9f3d70 con 0x5619ccb16f40 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f7999610640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccb16f40 0x5619ccb19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f797cff9640 10 client.15405 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.444+0000 7f7999610640 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccb16f40 0x5619ccb19360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2db84d1e874f83 server_cookie=4918be64ca95c820 in_seq=0 out_seq=0 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f797cff9640 1 -- 172.21.15.46:0/1476719066 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619cc9f3d70 con 0x5619ccb16f40 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f797cff9640 10 client.15405 handle_command_reply: tid=0 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 client.15405 shutdown 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 msgr2=0x5619ccaa77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5619cca84ec0 tx=0x7f7984010ce0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 shutdown_connections 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5619ccb16f40 0x5619ccb19360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f797cff9640 0 client.15405 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5619cca92cb0 0x5619ccaa4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619cca88410 0x5619ccaa77e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 --2- 172.21.15.46:0/1476719066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5619cca87af0 0x5619ccaa72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.450+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 >> 172.21.15.46:0/1476719066 conn(0x5619cca84060 msgr2=0x5619cca829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 shutdown_connections 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799e617740 1 -- 172.21.15.46:0/1476719066 wait complete. 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799e617740 20 client.15405 trim_cache size 0 max 16384 2024-09-12T02:51:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:51:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:51:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:51:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:51:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 msgr2=0x7f79800a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 0x7f79800a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f798c005fd0 tx=0x7f798c011810 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 msgr2=0x7f79941982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f79941982f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f799002a9a0 tx=0x7f7990004f70 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 shutdown_connections 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f79800a2ec0 0x7f79800a5380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.451+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79941069c0 0x7f799419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.452+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7994101350 0x7f7994198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.452+0000 7f799b874640 1 --2- 172.21.15.46:0/2236367535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79941009b0 0x7f79941982f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.452+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 >> 172.21.15.46:0/2236367535 conn(0x7f7994077990 msgr2=0x7f79940699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.452+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 shutdown_connections 2024-09-12T02:51:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:05.452+0000 7f799b874640 1 -- 172.21.15.46:0/2236367535 wait complete. 2024-09-12T02:51:05.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:05 smithi046 ceph-mon[30951]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T02:51:05.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:05 smithi145 ceph-mon[34674]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T02:51:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:05 smithi196 ceph-mon[34587]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T02:51:07.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:07 smithi046 ceph-mon[30951]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:07 smithi145 ceph-mon[34674]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:07 smithi196 ceph-mon[34587]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:09.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:09 smithi046 ceph-mon[30951]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:09.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:09 smithi145 ceph-mon[34674]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:09 smithi196 ceph-mon[34587]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:10.022 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 94% (12188/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T02:51:10.380 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T02:51:10.380 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.380 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.381 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-12T02:51:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T02:51:10.386 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T02:51:10.453 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T02:51:10.455 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:51:10.455 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T02:51:11.150 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T02:51:11.220 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T02:51:11.261 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:51:11.262 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:51:11.262 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T02:51:11.303 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:51:11.306 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T02:51:11.307 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T02:51:11.363 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-12T02:51:11.364 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-12T02:51:11.364 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T02:51:11.430 INFO:tasks.workunit.client.0.smithi046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T02:51:11.431 INFO:tasks.workunit.client.0.smithi046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-12T02:51:11.431 INFO:tasks.workunit.client.0.smithi046.stderr:+ trap -- retry EXIT 2024-09-12T02:51:11.431 INFO:tasks.workunit.client.0.smithi046.stderr:+ rm -rf ffsb 2024-09-12T02:51:11.432 INFO:tasks.workunit.client.0.smithi046.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-12T02:51:11.436 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into 'ffsb'... 2024-09-12T02:51:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:11 smithi145 ceph-mon[34674]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:11.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:11 smithi196 ceph-mon[34587]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:12.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:11 smithi046 ceph-mon[30951]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T02:51:13.086 INFO:tasks.workunit.client.0.smithi046.stderr:+ trap - EXIT 2024-09-12T02:51:13.086 INFO:tasks.workunit.client.0.smithi046.stderr:+ cd ffsb 2024-09-12T02:51:13.086 INFO:tasks.workunit.client.0.smithi046.stderr:+ ./configure 2024-09-12T02:51:13.213 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T02:51:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:13 smithi145 ceph-mon[34674]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:13 smithi196 ceph-mon[34587]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:14.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:13 smithi046 ceph-mon[30951]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T02:51:14.223 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether build environment is sane... yes 2024-09-12T02:51:14.241 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T02:51:14.241 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gawk... gawk 2024-09-12T02:51:14.269 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T02:51:14.317 INFO:tasks.workunit.client.0.smithi046.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T02:51:14.318 INFO:tasks.workunit.client.0.smithi046.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T02:51:14.339 INFO:tasks.workunit.client.0.smithi046.stdout:checking for style of include used by make... GNU 2024-09-12T02:51:14.344 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... gcc 2024-09-12T02:51:14.434 INFO:tasks.workunit.client.0.smithi046.stdout:checking for C compiler default output file name... a.out 2024-09-12T02:51:14.437 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether the C compiler works... yes 2024-09-12T02:51:14.440 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are cross compiling... no 2024-09-12T02:51:14.493 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of executables... 2024-09-12T02:51:14.538 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of object files... o 2024-09-12T02:51:14.582 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T02:51:14.625 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... yes 2024-09-12T02:51:14.708 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T02:51:14.857 INFO:tasks.workunit.client.0.smithi046.stdout:checking dependency style of gcc... gcc3 2024-09-12T02:51:14.952 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T02:51:15.025 INFO:tasks.workunit.client.0.smithi046.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T02:51:15.027 INFO:tasks.workunit.client.0.smithi046.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T02:51:15.042 INFO:tasks.workunit.client.0.smithi046.stdout:checking for AIX... no 2024-09-12T02:51:15.070 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... (cached) gcc 2024-09-12T02:51:15.084 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T02:51:15.085 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T02:51:15.085 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T02:51:15.086 INFO:tasks.workunit.client.0.smithi046.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T02:51:15.177 INFO:tasks.workunit.client.0.smithi046.stdout:checking for main in -lm... yes 2024-09-12T02:51:15.262 INFO:tasks.workunit.client.0.smithi046.stdout:checking for main in -lpthread... yes 2024-09-12T02:51:15.490 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ANSI C header files... yes 2024-09-12T02:51:15.563 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T02:51:15.624 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/types.h... yes 2024-09-12T02:51:15.688 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/stat.h... yes 2024-09-12T02:51:15.752 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdlib.h... yes 2024-09-12T02:51:15.822 INFO:tasks.workunit.client.0.smithi046.stdout:checking for string.h... yes 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:15.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:15.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:15 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.887 INFO:tasks.workunit.client.0.smithi046.stdout:checking for memory.h... yes 2024-09-12T02:51:15.955 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strings.h... yes 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:15.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:15 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.024 INFO:tasks.workunit.client.0.smithi046.stdout:checking for inttypes.h... yes 2024-09-12T02:51:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:51:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:16.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:15 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:51:16.084 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdint.h... yes 2024-09-12T02:51:16.152 INFO:tasks.workunit.client.0.smithi046.stdout:checking for unistd.h... yes 2024-09-12T02:51:16.240 INFO:tasks.workunit.client.0.smithi046.stdout:checking pthread.h usability... yes 2024-09-12T02:51:16.270 INFO:tasks.workunit.client.0.smithi046.stdout:checking pthread.h presence... yes 2024-09-12T02:51:16.271 INFO:tasks.workunit.client.0.smithi046.stdout:checking for pthread.h... yes 2024-09-12T02:51:16.342 INFO:tasks.workunit.client.0.smithi046.stdout:checking fcntl.h usability... yes 2024-09-12T02:51:16.371 INFO:tasks.workunit.client.0.smithi046.stdout:checking fcntl.h presence... yes 2024-09-12T02:51:16.372 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fcntl.h... yes 2024-09-12T02:51:16.450 INFO:tasks.workunit.client.0.smithi046.stdout:checking limits.h usability... yes 2024-09-12T02:51:16.483 INFO:tasks.workunit.client.0.smithi046.stdout:checking limits.h presence... yes 2024-09-12T02:51:16.483 INFO:tasks.workunit.client.0.smithi046.stdout:checking for limits.h... yes 2024-09-12T02:51:16.505 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdint.h... (cached) yes 2024-09-12T02:51:16.574 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h usability... yes 2024-09-12T02:51:16.603 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h presence... yes 2024-09-12T02:51:16.604 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/time.h... yes 2024-09-12T02:51:16.631 INFO:tasks.workunit.client.0.smithi046.stdout:checking for unistd.h... (cached) yes 2024-09-12T02:51:16.693 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/vfs.h usability... yes 2024-09-12T02:51:16.725 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/vfs.h presence... yes 2024-09-12T02:51:16.726 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/vfs.h... yes 2024-09-12T02:51:16.795 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/limits.h usability... no 2024-09-12T02:51:16.827 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/limits.h presence... no 2024-09-12T02:51:16.828 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/limits.h... no 2024-09-12T02:51:16.889 INFO:tasks.workunit.client.0.smithi046.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T02:51:16.937 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T02:51:16.939 INFO:tasks.workunit.client.0.smithi046.stdout:checking for function prototypes... yes 2024-09-12T02:51:16.986 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether setvbuf arguments are reversed... no 2024-09-12T02:51:17.075 INFO:tasks.workunit.client.0.smithi046.stdout:checking for system... yes 2024-09-12T02:51:17.166 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gettimeofday... yes 2024-09-12T02:51:17.268 INFO:tasks.workunit.client.0.smithi046.stdout:checking for mkdir... yes 2024-09-12T02:51:17.358 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strerror... yes 2024-09-12T02:51:17.448 INFO:tasks.workunit.client.0.smithi046.stdout:checking for open64... yes 2024-09-12T02:51:17.540 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stat64... yes 2024-09-12T02:51:17.633 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fseeko64... yes 2024-09-12T02:51:17.722 INFO:tasks.workunit.client.0.smithi046.stdout:checking for lrand48_r... yes 2024-09-12T02:51:17.811 INFO:tasks.workunit.client.0.smithi046.stdout:checking for srand48_r... yes 2024-09-12T02:51:17.839 INFO:tasks.workunit.client.0.smithi046.stdout:configure: creating ./config.status 2024-09-12T02:51:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:17 smithi145 ceph-mon[34674]: pgmap v36: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 362 KiB/s wr, 64 op/s 2024-09-12T02:51:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:17 smithi196 ceph-mon[34587]: pgmap v36: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 362 KiB/s wr, 64 op/s 2024-09-12T02:51:17.980 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating Makefile 2024-09-12T02:51:18.000 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating config.h 2024-09-12T02:51:18.027 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: executing depfiles commands 2024-09-12T02:51:18.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:17 smithi046 ceph-mon[30951]: pgmap v36: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 362 KiB/s wr, 64 op/s 2024-09-12T02:51:18.106 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: executing default commands 2024-09-12T02:51:18.164 INFO:tasks.workunit.client.0.smithi046.stderr:+ make 2024-09-12T02:51:18.193 INFO:tasks.workunit.client.0.smithi046.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-12T02:51:18.199 INFO:tasks.workunit.client.0.smithi046.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-12T02:51:18.201 INFO:tasks.workunit.client.0.smithi046.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-12T02:51:18.201 INFO:tasks.workunit.client.0.smithi046.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-12T02:51:18.201 INFO:tasks.workunit.client.0.smithi046.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-12T02:51:18.201 INFO:tasks.workunit.client.0.smithi046.stderr: any GNU archive site. 2024-09-12T02:51:18.206 INFO:tasks.workunit.client.0.smithi046.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-12T02:51:18.216 INFO:tasks.workunit.client.0.smithi046.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-12T02:51:18.218 INFO:tasks.workunit.client.0.smithi046.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-12T02:51:18.218 INFO:tasks.workunit.client.0.smithi046.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-12T02:51:18.218 INFO:tasks.workunit.client.0.smithi046.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-12T02:51:18.218 INFO:tasks.workunit.client.0.smithi046.stderr: Grab them from any GNU archive site. 2024-09-12T02:51:18.427 INFO:tasks.workunit.client.0.smithi046.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-12T02:51:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 -- 172.21.15.46:0/3038193826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a81035e0 msgr2=0x7f58a810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 --2- 172.21.15.46:0/3038193826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a81035e0 0x7f58a810bac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f589c009900 tx=0x7f589c02a480 comp rx=0 tx=0).stop 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 -- 172.21.15.46:0/3038193826 shutdown_connections 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 --2- 172.21.15.46:0/3038193826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a810c110 0x7f58a810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 --2- 172.21.15.46:0/3038193826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a81035e0 0x7f58a810bac0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 --2- 172.21.15.46:0/3038193826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a8102cc0 0x7f58a81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 -- 172.21.15.46:0/3038193826 >> 172.21.15.46:0/3038193826 conn(0x7f58a80fc6b0 msgr2=0x7f58a80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 -- 172.21.15.46:0/3038193826 shutdown_connections 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.540+0000 7f58adf11640 1 -- 172.21.15.46:0/3038193826 wait complete. 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 Processor -- start 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 -- start start 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a8102cc0 0x7f58a8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 0x7f58a81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111df0 con 0x7f58a810c110 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111c70 con 0x7f58a8102cc0 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58adf11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111f70 con 0x7f58a81035e0 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33444/0 (socket says 172.21.15.46:33444) 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58a7fff640 1 -- 172.21.15.46:0/2640626427 learned_addr learned my addr 172.21.15.46:0/2640626427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 0x7f58a81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.541+0000 7f58a77fe640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a8102cc0 0x7f58a8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 -- 172.21.15.46:0/2640626427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 msgr2=0x7f58a81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 0x7f58a81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 -- 172.21.15.46:0/2640626427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a8102cc0 msgr2=0x7f58a8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a8102cc0 0x7f58a8198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 -- 172.21.15.46:0/2640626427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a819d270 con 0x7f58a810c110 2024-09-12T02:51:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 0x7f58a81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a7fff640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f58a000eaa0 tx=0x7f58a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a4ff9640 1 -- 172.21.15.46:0/2640626427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a000cdb0 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a4ff9640 1 -- 172.21.15.46:0/2640626427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58a0005020 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58a4ff9640 1 -- 172.21.15.46:0/2640626427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00116d0 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a819d560 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.542+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81a4df0 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.545+0000 7f58a4ff9640 1 -- 172.21.15.46:0/2640626427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f58a0004b20 con 0x7f58a810c110 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.545+0000 7f58b0cac740 1 Processor -- start 2024-09-12T02:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.545+0000 7f58b0cac740 1 -- start start 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.545+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573504faf0 0x55573504fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.545+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573505acb0 0x55573505d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58b0cac740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555734fb42f0 con 0x555735050410 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58b0cac740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555734f5c270 con 0x55573504faf0 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58b0cac740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555734fb02b0 con 0x55573505acb0 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a4ff9640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 0x7f58900a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33452/0 (socket says 172.21.15.46:33452) 2024-09-12T02:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 -- 172.21.15.46:0/744417911 learned_addr learned my addr 172.21.15.46:0/744417911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573504faf0 0x55573504fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573505acb0 0x55573505d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a7fff640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 0x7f58900a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a4ff9640 1 -- 172.21.15.46:0/2640626427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f58a00cfb10 con 0x7f58a810c110 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 -- 172.21.15.46:0/744417911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573505acb0 msgr2=0x55573505d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573505acb0 0x55573505d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 -- 172.21.15.46:0/744417911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573504faf0 msgr2=0x55573504fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573504faf0 0x55573504fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 -- 172.21.15.46:0/744417911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555734f5b340 con 0x555735050410 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a77fe640 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f589800bd10 tx=0x7f58980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff76a5238ee0a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f58a7fff640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 0x7f58900a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58a00ccde0 tx=0x7f58a0090fc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.546+0000 7f588dffb640 1 -- 172.21.15.46:0/744417911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898004030 con 0x555735050410 2024-09-12T02:51:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f588dffb640 1 -- 172.21.15.46:0/744417911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58980035f0 con 0x555735050410 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f588dffb640 1 -- 172.21.15.46:0/744417911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58980116a0 con 0x555735050410 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 -- 172.21.15.46:0/744417911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 msgr2=0x55573505a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f589800bd10 tx=0x7f58980107a0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 -- 172.21.15.46:0/744417911 shutdown_connections 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573505acb0 0x55573505d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x55573505a770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 --2- 172.21.15.46:0/744417911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573504faf0 0x55573504fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 -- 172.21.15.46:0/744417911 >> 172.21.15.46:0/744417911 conn(0x55573504c060 msgr2=0x55573504c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 -- 172.21.15.46:0/744417911 shutdown_connections 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.547+0000 7f58b0cac740 1 -- 172.21.15.46:0/744417911 wait complete. 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 -- start start 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 Processor -- start 2024-09-12T02:51:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735057b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x5557350585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55573505d880 con 0x555735050410 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555734f3d290 con 0x55573505acb0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55573505da00 con 0x55573504faf0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58b0cac740 10 client.0 _ll_register_callbacks cb 0x555734f6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33460/0 (socket says 172.21.15.46:33460) 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58a77fe640 1 -- 172.21.15.46:0/380641544 learned_addr learned my addr 172.21.15.46:0/380641544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735057b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.548+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x5557350585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 -- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 msgr2=0x555735057b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735057b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 -- 172.21.15.46:0/380641544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 msgr2=0x5557350585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x5557350585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 -- 172.21.15.46:0/380641544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555734cc7210 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735057b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58a77fe640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f589800b890 tx=0x7f589800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695101266502bce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898029060 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f5898003a30 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58980035f0 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5898030050 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58b0cac740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555734d93620 con 0x555735050410 2024-09-12T02:51:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58980035f0 con 0x555735050410 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f58980046c0 con 0x555735050410 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58b0cac740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.549+0000 7f58b0cac740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58b0cac740 20 client.15417 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55573506ab40 0x55573506cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58b0cac740 4 client.15417 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555734fc1a20 con 0x55573506ab40 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55573506ab40 0x55573506cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55573506ab40 0x55573506cf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0936fa8fcc4842a server_cookie=186735c45e5972fc in_seq=0 out_seq=0 2024-09-12T02:51:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.550+0000 7f5886ffd640 10 client.15417 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f5886ffd640 1 -- 172.21.15.46:0/380641544 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555734fc1a20 con 0x55573506ab40 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f5886ffd640 10 client.15417 handle_command_reply: tid=0 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 client.15417 shutdown 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 msgr2=0x555735058080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f589800b890 tx=0x7f589800b8c0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 shutdown_connections 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55573506ab40 0x55573506cf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x5557350585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735058080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 --2- 172.21.15.46:0/380641544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735057b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f5886ffd640 0 client.15417 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 >> 172.21.15.46:0/380641544 conn(0x55573504c060 msgr2=0x555735051dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 shutdown_connections 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 1 -- 172.21.15.46:0/380641544 wait complete. 2024-09-12T02:51:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.551+0000 7f58b0cac740 20 client.15417 trim_cache size 0 max 16384 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.565+0000 7f58b0cac740 1 Processor -- start 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.565+0000 7f58b0cac740 1 -- start start 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555735050410 0x555735072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81b1700 con 0x55573504faf0 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81aefd0 con 0x55573505acb0 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58b0cac740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81ace20 con 0x555735050410 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33466/0 (socket says 172.21.15.46:33466) 2024-09-12T02:51:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 -- 172.21.15.46:0/3590733823 learned_addr learned my addr 172.21.15.46:0/3590733823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a77fe640 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555735050410 0x555735072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 -- 172.21.15.46:0/3590733823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555735050410 msgr2=0x555735072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555735050410 0x555735072500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 -- 172.21.15.46:0/3590733823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 msgr2=0x555735072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 -- 172.21.15.46:0/3590733823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55573504be90 con 0x55573504faf0 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f589c009230 tx=0x7f589c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3100464cce0e6ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f588d7fa640 1 -- 172.21.15.46:0/3590733823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c035040 con 0x55573504faf0 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f588d7fa640 1 -- 172.21.15.46:0/3590733823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c004c20 con 0x55573504faf0 2024-09-12T02:51:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.566+0000 7f588d7fa640 1 -- 172.21.15.46:0/3590733823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c003dc0 con 0x55573504faf0 2024-09-12T02:51:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- 172.21.15.46:0/3590733823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 msgr2=0x5557350583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f589c009230 tx=0x7f589c009260 comp rx=0 tx=0).stop 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- 172.21.15.46:0/3590733823 shutdown_connections 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555735050410 0x555735072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 --2- 172.21.15.46:0/3590733823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55573504faf0 0x5557350583f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- 172.21.15.46:0/3590733823 >> 172.21.15.46:0/3590733823 conn(0x55573504c060 msgr2=0x555735059bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- 172.21.15.46:0/3590733823 shutdown_connections 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- 172.21.15.46:0/3590733823 wait complete. 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.567+0000 7f58b0cac740 1 -- start start 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 Processor -- start 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735063320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735054d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81ad490 con 0x555735050410 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81ae900 con 0x55573505acb0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81adb60 con 0x55573504faf0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58b0cac740 10 client.0 _ll_register_callbacks cb 0x7f589c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33472/0 (socket says 172.21.15.46:33472) 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735063320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 -- 172.21.15.46:0/4102635609 learned_addr learned my addr 172.21.15.46:0/4102635609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735054d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 -- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 msgr2=0x555735063320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735063320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 -- 172.21.15.46:0/4102635609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 msgr2=0x555735054d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735054d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a77fe640 1 -- 172.21.15.46:0/4102635609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555735052940 con 0x555735050410 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.568+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735063320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58a77fe640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5898005d00 tx=0x7f5898010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9577997368d1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898039060 con 0x555735050410 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f589800b390 con 0x555735050410 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589c004c20 con 0x555735050410 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5898040050 con 0x555735050410 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 10 client.15423 fetch_fsmap learned FSMap version 25 2024-09-12T02:51:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557350d52d0 con 0x555735050410 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c004c20 con 0x555735050410 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f58980115d0 con 0x555735050410 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 10 client.15423 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 20 client.15423 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5557350d5900 0x5557350d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 4 client.15423 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.569+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555734fbbd70 con 0x5557350d5900 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.570+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5557350d5900 0x5557350d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.570+0000 7f5886ffd640 10 client.15423 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.570+0000 7f58a6ffd640 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5557350d5900 0x5557350d7d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca4080ff78d5350 server_cookie=a9e05cce6a5b3aa2 in_seq=0 out_seq=0 2024-09-12T02:51:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f5886ffd640 1 -- 172.21.15.46:0/4102635609 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555734fbbd70 con 0x5557350d5900 2024-09-12T02:51:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f5886ffd640 10 client.15423 handle_command_reply: tid=0 2024-09-12T02:51:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 client.15423 shutdown 2024-09-12T02:51:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 msgr2=0x555735054810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5898005d00 tx=0x7f5898010d20 comp rx=0 tx=0).stop 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 shutdown_connections 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5557350d5900 0x5557350d7d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573505acb0 0x555735054d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f5886ffd640 0 client.15423 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555735050410 0x555735054810 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 --2- 172.21.15.46:0/4102635609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55573504faf0 0x555735063320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 >> 172.21.15.46:0/4102635609 conn(0x55573504c060 msgr2=0x55573504a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 shutdown_connections 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.689+0000 7f58b0cac740 1 -- 172.21.15.46:0/4102635609 wait complete. 2024-09-12T02:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58b0cac740 20 client.15423 trim_cache size 0 max 16384 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 msgr2=0x7f58900a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 0x7f58900a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58a00ccde0 tx=0x7f58a0090fc0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 msgr2=0x7f58a819cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f58a000eaa0 tx=0x7f58a000ef60 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 shutdown_connections 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f58900a2ec0 0x7f58900a5380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a810c110 0x7f58a819cb40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58a81035e0 0x7f58a81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 --2- 172.21.15.46:0/2640626427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58a8102cc0 0x7f58a8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 >> 172.21.15.46:0/2640626427 conn(0x7f58a80fc6b0 msgr2=0x7f58a80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 shutdown_connections 2024-09-12T02:51:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:18.690+0000 7f58adf11640 1 -- 172.21.15.46:0/2640626427 wait complete. 2024-09-12T02:51:18.761 INFO:tasks.workunit.client.0.smithi046.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-12T02:51:18.761 INFO:tasks.workunit.client.0.smithi046.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-12T02:51:18.761 INFO:tasks.workunit.client.0.smithi046.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-12T02:51:18.761 INFO:tasks.workunit.client.0.smithi046.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-12T02:51:18.894 INFO:tasks.workunit.client.0.smithi046.stdout:/bin/sh ./config.status --recheck 2024-09-12T02:51:18.955 INFO:tasks.workunit.client.0.smithi046.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-12T02:51:19.104 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T02:51:19.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:19 smithi145 ceph-mon[34674]: pgmap v37: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:19 smithi196 ceph-mon[34587]: pgmap v37: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:20.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:19 smithi046 ceph-mon[30951]: pgmap v37: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:20.114 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether build environment is sane... yes 2024-09-12T02:51:20.125 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T02:51:20.126 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gawk... gawk 2024-09-12T02:51:20.153 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T02:51:20.202 INFO:tasks.workunit.client.0.smithi046.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T02:51:20.202 INFO:tasks.workunit.client.0.smithi046.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T02:51:20.239 INFO:tasks.workunit.client.0.smithi046.stdout:checking for style of include used by make... GNU 2024-09-12T02:51:20.244 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... gcc 2024-09-12T02:51:20.366 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether the C compiler works... yes 2024-09-12T02:51:20.366 INFO:tasks.workunit.client.0.smithi046.stdout:checking for C compiler default output file name... a.out 2024-09-12T02:51:20.419 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of executables... 2024-09-12T02:51:20.474 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are cross compiling... no 2024-09-12T02:51:20.522 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of object files... o 2024-09-12T02:51:20.556 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T02:51:20.590 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... yes 2024-09-12T02:51:20.634 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T02:51:20.784 INFO:tasks.workunit.client.0.smithi046.stdout:checking dependency style of gcc... gcc3 2024-09-12T02:51:20.850 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T02:51:20.918 INFO:tasks.workunit.client.0.smithi046.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T02:51:20.920 INFO:tasks.workunit.client.0.smithi046.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T02:51:21.089 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ANSI C header files... yes 2024-09-12T02:51:21.137 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/types.h... yes 2024-09-12T02:51:21.193 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/stat.h... yes 2024-09-12T02:51:21.256 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdlib.h... yes 2024-09-12T02:51:21.309 INFO:tasks.workunit.client.0.smithi046.stdout:checking for string.h... yes 2024-09-12T02:51:21.365 INFO:tasks.workunit.client.0.smithi046.stdout:checking for memory.h... yes 2024-09-12T02:51:21.423 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strings.h... yes 2024-09-12T02:51:21.480 INFO:tasks.workunit.client.0.smithi046.stdout:checking for inttypes.h... yes 2024-09-12T02:51:21.535 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdint.h... yes 2024-09-12T02:51:21.593 INFO:tasks.workunit.client.0.smithi046.stdout:checking for unistd.h... yes 2024-09-12T02:51:21.653 INFO:tasks.workunit.client.0.smithi046.stdout:checking minix/config.h usability... no 2024-09-12T02:51:21.695 INFO:tasks.workunit.client.0.smithi046.stdout:checking minix/config.h presence... no 2024-09-12T02:51:21.695 INFO:tasks.workunit.client.0.smithi046.stdout:checking for minix/config.h... no 2024-09-12T02:51:21.739 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-12T02:51:21.749 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... (cached) gcc 2024-09-12T02:51:21.802 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T02:51:21.803 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T02:51:21.803 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T02:51:21.803 INFO:tasks.workunit.client.0.smithi046.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T02:51:21.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:21 smithi145 ceph-mon[34674]: pgmap v38: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:21.887 INFO:tasks.workunit.client.0.smithi046.stdout:checking for main in -lm... yes 2024-09-12T02:51:21.954 INFO:tasks.workunit.client.0.smithi046.stdout:checking for main in -lpthread... yes 2024-09-12T02:51:21.958 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ANSI C header files... (cached) yes 2024-09-12T02:51:21.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:21 smithi196 ceph-mon[34587]: pgmap v38: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:22.000 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T02:51:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:21 smithi046 ceph-mon[30951]: pgmap v38: 129 pgs: 129 active+clean; 5.2 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 362 KiB/s wr, 61 op/s 2024-09-12T02:51:22.057 INFO:tasks.workunit.client.0.smithi046.stdout:checking pthread.h usability... yes 2024-09-12T02:51:22.082 INFO:tasks.workunit.client.0.smithi046.stdout:checking pthread.h presence... yes 2024-09-12T02:51:22.082 INFO:tasks.workunit.client.0.smithi046.stdout:checking for pthread.h... yes 2024-09-12T02:51:22.145 INFO:tasks.workunit.client.0.smithi046.stdout:checking fcntl.h usability... yes 2024-09-12T02:51:22.169 INFO:tasks.workunit.client.0.smithi046.stdout:checking fcntl.h presence... yes 2024-09-12T02:51:22.169 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fcntl.h... yes 2024-09-12T02:51:22.280 INFO:tasks.workunit.client.0.smithi046.stdout:checking limits.h usability... yes 2024-09-12T02:51:22.303 INFO:tasks.workunit.client.0.smithi046.stdout:checking limits.h presence... yes 2024-09-12T02:51:22.303 INFO:tasks.workunit.client.0.smithi046.stdout:checking for limits.h... yes 2024-09-12T02:51:22.325 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdint.h... (cached) yes 2024-09-12T02:51:22.378 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h usability... yes 2024-09-12T02:51:22.403 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h presence... yes 2024-09-12T02:51:22.403 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/time.h... yes 2024-09-12T02:51:22.425 INFO:tasks.workunit.client.0.smithi046.stdout:checking for unistd.h... (cached) yes 2024-09-12T02:51:22.481 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/vfs.h usability... yes 2024-09-12T02:51:22.507 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/vfs.h presence... yes 2024-09-12T02:51:22.508 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/vfs.h... yes 2024-09-12T02:51:22.592 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/limits.h usability... no 2024-09-12T02:51:22.673 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/limits.h presence... no 2024-09-12T02:51:22.673 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/limits.h... no 2024-09-12T02:51:22.707 INFO:tasks.workunit.client.0.smithi046.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T02:51:22.744 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T02:51:22.829 INFO:tasks.workunit.client.0.smithi046.stdout:checking for system... yes 2024-09-12T02:51:22.909 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gettimeofday... yes 2024-09-12T02:51:22.990 INFO:tasks.workunit.client.0.smithi046.stdout:checking for mkdir... yes 2024-09-12T02:51:23.069 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strerror... yes 2024-09-12T02:51:23.149 INFO:tasks.workunit.client.0.smithi046.stdout:checking for open64... yes 2024-09-12T02:51:23.229 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stat64... yes 2024-09-12T02:51:23.356 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fseeko64... yes 2024-09-12T02:51:23.438 INFO:tasks.workunit.client.0.smithi046.stdout:checking for lrand48_r... yes 2024-09-12T02:51:23.519 INFO:tasks.workunit.client.0.smithi046.stdout:checking for srand48_r... yes 2024-09-12T02:51:23.547 INFO:tasks.workunit.client.0.smithi046.stdout:configure: creating ./config.status 2024-09-12T02:51:23.655 INFO:tasks.workunit.client.0.smithi046.stdout: /bin/sh ./config.status 2024-09-12T02:51:23.719 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating Makefile 2024-09-12T02:51:23.737 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating config.h 2024-09-12T02:51:23.764 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: executing depfiles commands 2024-09-12T02:51:23.784 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: executing default commands 2024-09-12T02:51:23.836 INFO:tasks.workunit.client.0.smithi046.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-12T02:51:23.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:23 smithi145 ceph-mon[34674]: pgmap v39: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 677 KiB/s wr, 232 op/s 2024-09-12T02:51:23.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:23 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:51:23.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:23 smithi196 ceph-mon[34587]: pgmap v39: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 677 KiB/s wr, 232 op/s 2024-09-12T02:51:23.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:23 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:51:24.005 INFO:tasks.workunit.client.0.smithi046.stdout:rm -f stamp-h1 2024-09-12T02:51:24.008 INFO:tasks.workunit.client.0.smithi046.stdout:touch config.h.in 2024-09-12T02:51:24.010 INFO:tasks.workunit.client.0.smithi046.stdout:cd . && /bin/sh ./config.status config.h 2024-09-12T02:51:24.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:23 smithi046 ceph-mon[30951]: pgmap v39: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 677 KiB/s wr, 232 op/s 2024-09-12T02:51:24.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:23 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:51:24.065 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating config.h 2024-09-12T02:51:24.102 INFO:tasks.workunit.client.0.smithi046.stdout:make all-am 2024-09-12T02:51:24.129 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T02:51:24.129 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-12T02:51:24.319 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-12T02:51:24.324 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-12T02:51:24.430 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-12T02:51:24.435 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-12T02:51:24.542 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-12T02:51:24.547 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-12T02:51:24.663 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-12T02:51:24.669 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-12T02:51:24.822 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-12T02:51:24.828 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-12T02:51:24.901 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-12T02:51:24.906 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-12T02:51:24.962 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-12T02:51:24.967 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-12T02:51:25.030 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-12T02:51:25.035 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-12T02:51:25.270 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-12T02:51:25.276 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-12T02:51:25.431 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-12T02:51:25.436 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-12T02:51:25.626 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-12T02:51:25.631 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-12T02:51:25.718 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-12T02:51:25.723 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-12T02:51:25.820 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-12T02:51:25.825 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-12T02:51:25.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:25 smithi145 ceph-mon[34674]: pgmap v40: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 677 KiB/s wr, 229 op/s 2024-09-12T02:51:25.958 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-12T02:51:25.963 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-12T02:51:25.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:25 smithi196 ceph-mon[34587]: pgmap v40: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 677 KiB/s wr, 229 op/s 2024-09-12T02:51:26.008 INFO:tasks.workunit.client.0.smithi046.stderr:In file included from parser.h:22, 2024-09-12T02:51:26.008 INFO:tasks.workunit.client.0.smithi046.stderr: from parser.c:25: 2024-09-12T02:51:26.008 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c: In function ‘init_filesys’: 2024-09-12T02:51:26.009 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.009 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.009 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.009 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.009 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.010 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.011 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T02:51:26.012 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.013 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.014 INFO:tasks.workunit.client.0.smithi046.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T02:51:26.014 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.014 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-12T02:51:26.014 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.015 INFO:tasks.workunit.client.0.smithi046.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:51:26.016 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.017 INFO:tasks.workunit.client.0.smithi046.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:51:26.018 INFO:tasks.workunit.client.0.smithi046.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~ 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T02:51:26.019 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:25 smithi046 ceph-mon[30951]: pgmap v40: 129 pgs: 129 active+clean; 3.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 677 KiB/s wr, 229 op/s 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c: In function ‘get_optstr’: 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:200:14: note: source object declared here 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr: 200 | char temp[BUFSIZE]; 2024-09-12T02:51:26.058 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~ 2024-09-12T02:51:26.064 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c: In function ‘size64_convert’: 2024-09-12T02:51:26.064 INFO:tasks.workunit.client.0.smithi046.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-12T02:51:26.064 INFO:tasks.workunit.client.0.smithi046.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-12T02:51:26.064 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-12T02:51:26.381 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-12T02:51:26.387 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-12T02:51:26.468 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-12T02:51:26.473 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-12T02:51:26.585 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-12T02:51:26.592 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-12T02:51:26.634 INFO:tasks.workunit.client.0.smithi046.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-12T02:51:26.641 INFO:tasks.workunit.client.0.smithi046.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-12T02:51:26.748 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T02:51:26.748 INFO:tasks.workunit.client.0.smithi046.stderr:+ cd .. 2024-09-12T02:51:26.750 INFO:tasks.workunit.client.0.smithi046.stderr:+ mkdir tmp 2024-09-12T02:51:26.753 INFO:tasks.workunit.client.0.smithi046.stderr:+ cd tmp 2024-09-12T02:51:26.754 INFO:tasks.workunit.client.0.smithi046.stderr:+ for f in $mydir/*.ffsb 2024-09-12T02:51:26.754 INFO:tasks.workunit.client.0.smithi046.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-12T02:51:26.754 INFO:tasks.workunit.client.0.smithi046.stdout:FFSB version 6.0-RC2 started 2024-09-12T02:51:26.754 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout:benchmark time = 300 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout:ThreadGroup 0 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout:================ 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: num_threads = 32 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: read_random = off 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: read_size = 0 (0B) 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: read_blocksize = 0 (0B) 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: read_skip = off 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: read_skipsize = 0 (0B) 2024-09-12T02:51:26.758 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: write_random = on 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: write_size = 5242880 (5MB) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: fsync_file = 0 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: write_blocksize = 4096 (4KB) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: wait time = 0 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: op weights 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: read = 0 (0.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: readall = 0 (0.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: write = 1 (100.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: create = 0 (0.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: append = 0 (0.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: delete = 0 (0.00%) 2024-09-12T02:51:26.759 INFO:tasks.workunit.client.0.smithi046.stdout: metaop = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: createdir = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: stat = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: writeall = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: writeall_fsync = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: open_close = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: write_fsync = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: create_fsync = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: append_fsync = 0 (0.00%) 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout:FileSystem . 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout:========== 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: num_dirs = 0 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: starting files = 128 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.760 INFO:tasks.workunit.client.0.smithi046.stdout: min file size = 104857600 (100MB) 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: max file size = 104857600 (100MB) 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: directio = off 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: alignedio = on 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: bufferedio = off 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: aging is off 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: current utilization = 0.04% 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout:checking existing fs: . 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stdout:recreating new fileset 2024-09-12T02:51:26.761 INFO:tasks.workunit.client.0.smithi046.stderr:opendir: No such file or directory 2024-09-12T02:51:27.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:27 smithi145 ceph-mon[34674]: pgmap v41: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 942 KiB/s wr, 289 op/s 2024-09-12T02:51:27.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:27 smithi196 ceph-mon[34587]: pgmap v41: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 942 KiB/s wr, 289 op/s 2024-09-12T02:51:28.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:27 smithi046 ceph-mon[30951]: pgmap v41: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 942 KiB/s wr, 289 op/s 2024-09-12T02:51:29.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:29 smithi145 ceph-mon[34674]: pgmap v42: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:29 smithi196 ceph-mon[34587]: pgmap v42: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:30.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:29 smithi046 ceph-mon[30951]: pgmap v42: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.788+0000 7faeed047640 1 -- 172.21.15.46:0/2112852458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 msgr2=0x7faee810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.788+0000 7faeed047640 1 --2- 172.21.15.46:0/2112852458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee810d880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faee0009cd0 tx=0x7faee002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.788+0000 7faeed047640 1 -- 172.21.15.46:0/2112852458 shutdown_connections 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 --2- 172.21.15.46:0/2112852458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee810d880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 --2- 172.21.15.46:0/2112852458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81009b0 0x7faee8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 --2- 172.21.15.46:0/2112852458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81069c0 0x7faee8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 -- 172.21.15.46:0/2112852458 >> 172.21.15.46:0/2112852458 conn(0x7faee8077990 msgr2=0x7faee80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 -- 172.21.15.46:0/2112852458 shutdown_connections 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 -- 172.21.15.46:0/2112852458 wait complete. 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.789+0000 7faeed047640 1 Processor -- start 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 -- start start 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81009b0 0x7faee819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81069c0 0x7faee819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee8110d00 con 0x7faee8101350 2024-09-12T02:51:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faee8110b80 con 0x7faee81009b0 2024-09-12T02:51:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faeed047640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faee8110e80 con 0x7faee81069c0 2024-09-12T02:51:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47674/0 (socket says 172.21.15.46:47674) 2024-09-12T02:51:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 -- 172.21.15.46:0/4270100841 learned_addr learned my addr 172.21.15.46:0/4270100841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee6d9d640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81009b0 0x7faee819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee759e640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81069c0 0x7faee819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 -- 172.21.15.46:0/4270100841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81069c0 msgr2=0x7faee819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81069c0 0x7faee819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 -- 172.21.15.46:0/4270100841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81009b0 msgr2=0x7faee819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81009b0 0x7faee819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.790+0000 7faee659c640 1 -- 172.21.15.46:0/4270100841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee819f400 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faee659c640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faedc00d950 tx=0x7faedc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faecffff640 1 -- 172.21.15.46:0/4270100841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc017070 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faecffff640 1 -- 172.21.15.46:0/4270100841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faedc009c30 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee819f6f0 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faecffff640 1 -- 172.21.15.46:0/4270100841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00f620 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.791+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee81a6f80 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.793+0000 7faecffff640 1 -- 172.21.15.46:0/4270100841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7faedc0109a0 con 0x7faee8101350 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 Processor -- start 2024-09-12T02:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faecffff640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 0x7faed00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 -- start start 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faee759e640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 0x7faed00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faecffff640 1 -- 172.21.15.46:0/4270100841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7faedc0cf5f0 con 0x7faee8101350 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 0x557088ba1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 0x557088baf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557088b062f0 con 0x557088ba2410 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557088aae270 con 0x557088ba1af0 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.794+0000 7faeefde0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557088b022b0 con 0x557088baccb0 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47678/0 (socket says 172.21.15.46:47678) 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 -- 172.21.15.46:0/1194918350 learned_addr learned my addr 172.21.15.46:0/1194918350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee6d9d640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 0x557088ba1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee759e640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 0x557088baf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 -- 172.21.15.46:0/1194918350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 msgr2=0x557088baf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 0x557088baf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 -- 172.21.15.46:0/1194918350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 msgr2=0x557088ba1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 0x557088ba1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 -- 172.21.15.46:0/1194918350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557088aad340 con 0x557088ba2410 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee6d9d640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 0x557088ba1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee759e640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 0x557088baf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee759e640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 0x7faed00a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faee0004af0 tx=0x7faee0005d80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faee659c640 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faedc097a70 tx=0x7faedc009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39495038133e963f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faecd7fa640 1 -- 172.21.15.46:0/1194918350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc020040 con 0x557088ba2410 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faecd7fa640 1 -- 172.21.15.46:0/1194918350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faedc09f910 con 0x557088ba2410 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.795+0000 7faecd7fa640 1 -- 172.21.15.46:0/1194918350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc0ccb40 con 0x557088ba2410 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- 172.21.15.46:0/1194918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 msgr2=0x557088bac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faedc097a70 tx=0x7faedc009de0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- 172.21.15.46:0/1194918350 shutdown_connections 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088baccb0 0x557088baf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba2410 0x557088bac770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 --2- 172.21.15.46:0/1194918350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088ba1af0 0x557088ba1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- 172.21.15.46:0/1194918350 >> 172.21.15.46:0/1194918350 conn(0x557088b9e060 msgr2=0x557088b9e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- 172.21.15.46:0/1194918350 shutdown_connections 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- 172.21.15.46:0/1194918350 wait complete. 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.796+0000 7faeefde0740 1 -- start start 2024-09-12T02:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 Processor -- start 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088baa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557088baf880 con 0x557088ba1af0 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557088a8f290 con 0x557088baccb0 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557088bafa00 con 0x557088ba2410 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faeefde0740 10 client.0 _ll_register_callbacks cb 0x557088ac0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47694/0 (socket says 172.21.15.46:47694) 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 -- 172.21.15.46:0/1201023680 learned_addr learned my addr 172.21.15.46:0/1201023680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee659c640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee759e640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088baa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 -- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 msgr2=0x557088baa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 -- 172.21.15.46:0/1201023680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 msgr2=0x557088baa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088baa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.797+0000 7faee6d9d640 1 -- 172.21.15.46:0/1201023680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557088819210 con 0x557088ba1af0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faee6d9d640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faed800ba30 tx=0x7faed800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf7586f71c411e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8011440 con 0x557088ba1af0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7faed80115e0 con 0x557088ba1af0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faedc09f910 con 0x557088ba1af0 2024-09-12T02:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faed80117e0 con 0x557088ba1af0 2024-09-12T02:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faeefde0740 10 client.15435 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570888e5620 con 0x557088ba1af0 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faedc09f910 con 0x557088ba1af0 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7faed801b950 con 0x557088ba1af0 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faeefde0740 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.798+0000 7faeefde0740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faeefde0740 20 client.15435 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088bbcc10 0x557088bbf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faeefde0740 4 client.15435 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557088b13a20 con 0x557088bbcc10 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faee659c640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088bbcc10 0x557088bbf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faecaff5640 10 client.15435 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.799+0000 7faee659c640 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088bbcc10 0x557088bbf000 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99785169ee732f9b server_cookie=4861a634e4487b2a in_seq=0 out_seq=0 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faecaff5640 1 -- 172.21.15.46:0/1201023680 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557088b13a20 con 0x557088bbcc10 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faecaff5640 10 client.15435 handle_command_reply: tid=0 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 client.15435 shutdown 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 msgr2=0x557088ba9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faed800ba30 tx=0x7faed800bef0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 shutdown_connections 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088bbcc10 0x557088bbf000 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088baa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 --2- 172.21.15.46:0/1201023680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba9b90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 >> 172.21.15.46:0/1201023680 conn(0x557088b9e060 msgr2=0x557088ba3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faecaff5640 0 client.15435 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 shutdown_connections 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 1 -- 172.21.15.46:0/1201023680 wait complete. 2024-09-12T02:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.800+0000 7faeefde0740 20 client.15435 trim_cache size 0 max 16384 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.815+0000 7faeefde0740 1 Processor -- start 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.815+0000 7faeefde0740 1 -- start start 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088b9d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b3890 con 0x557088ba1af0 2024-09-12T02:51:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b1160 con 0x557088baccb0 2024-09-12T02:51:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeefde0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faee81aefb0 con 0x557088ba2410 2024-09-12T02:51:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47700/0 (socket says 172.21.15.46:47700) 2024-09-12T02:51:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 -- 172.21.15.46:0/240067142 learned_addr learned my addr 172.21.15.46:0/240067142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 -- 172.21.15.46:0/240067142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 msgr2=0x557088baa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 -- 172.21.15.46:0/240067142 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 msgr2=0x557088b9d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088b9d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 -- 172.21.15.46:0/240067142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557088b9de90 con 0x557088ba1af0 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faee6d9d640 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faed801bdc0 tx=0x7faed8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ad40a03828e0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeccff9640 1 -- 172.21.15.46:0/240067142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8011440 con 0x557088ba1af0 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeccff9640 1 -- 172.21.15.46:0/240067142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faed802e070 con 0x557088ba1af0 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.816+0000 7faeccff9640 1 -- 172.21.15.46:0/240067142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed801b420 con 0x557088ba1af0 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- 172.21.15.46:0/240067142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 msgr2=0x557088ba4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faed801bdc0 tx=0x7faed8003ee0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- 172.21.15.46:0/240067142 shutdown_connections 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088b9d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088baa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 --2- 172.21.15.46:0/240067142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088ba4560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- 172.21.15.46:0/240067142 >> 172.21.15.46:0/240067142 conn(0x557088b9e060 msgr2=0x557088b9c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- 172.21.15.46:0/240067142 shutdown_connections 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- 172.21.15.46:0/240067142 wait complete. 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.817+0000 7faeefde0740 1 -- start start 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 Processor -- start 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088bb0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088bb06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee81af620 con 0x557088ba1af0 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b0a90 con 0x557088baccb0 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faee81afcf0 con 0x557088ba2410 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faeefde0740 10 client.0 _ll_register_callbacks cb 0x7faed8004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47712/0 (socket says 172.21.15.46:47712) 2024-09-12T02:51:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 -- 172.21.15.46:0/3582303702 learned_addr learned my addr 172.21.15.46:0/3582303702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee659c640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088bb0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee759e640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088bb06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 -- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 msgr2=0x557088bb0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088bb0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 -- 172.21.15.46:0/3582303702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 msgr2=0x557088bb06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088bb06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee6d9d640 1 -- 172.21.15.46:0/3582303702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557088c25980 con 0x557088ba1af0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee659c640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088bb0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.818+0000 7faee759e640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088bb06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faee6d9d640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faed8005290 tx=0x7faed8005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0fae0e789a6b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8036070 con 0x557088ba1af0 2024-09-12T02:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7faed8011b90 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed802e070 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faed8011d90 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557088c26c90 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faed802e070 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7faed8029680 con 0x557088ba1af0 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 20 client.15441 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088c27140 0x557088c29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 4 client.15441 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.819+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557088b0dd70 con 0x557088c27140 2024-09-12T02:51:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.820+0000 7faee659c640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088c27140 0x557088c29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.820+0000 7faecaff5640 10 client.15441 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.820+0000 7faee659c640 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088c27140 0x557088c29560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f60d6da5ff1b590 server_cookie=de74f29b25f72820 in_seq=0 out_seq=0 2024-09-12T02:51:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.855+0000 7faecaff5640 1 -- 172.21.15.46:0/3582303702 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557088b0dd70 con 0x557088c27140 2024-09-12T02:51:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.855+0000 7faecaff5640 10 client.15441 handle_command_reply: tid=0 2024-09-12T02:51:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 client.15441 shutdown 2024-09-12T02:51:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 msgr2=0x557088bafc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faed8005290 tx=0x7faed8005390 comp rx=0 tx=0).stop 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 shutdown_connections 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557088c27140 0x557088c29560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557088baccb0 0x557088bb06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557088ba2410 0x557088bb0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faecaff5640 0 client.15441 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 --2- 172.21.15.46:0/3582303702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557088ba1af0 0x557088bafc50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 >> 172.21.15.46:0/3582303702 conn(0x557088b9e060 msgr2=0x557088b9d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 shutdown_connections 2024-09-12T02:51:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 1 -- 172.21.15.46:0/3582303702 wait complete. 2024-09-12T02:51:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.856+0000 7faeefde0740 20 client.15441 trim_cache size 0 max 16384 2024-09-12T02:51:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:51:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:51:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:51:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:51:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 msgr2=0x7faed00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 0x7faed00a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faee0004af0 tx=0x7faee0005d80 comp rx=0 tx=0).stop 2024-09-12T02:51:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 msgr2=0x7faee819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faedc00d950 tx=0x7faedc00de10 comp rx=0 tx=0).stop 2024-09-12T02:51:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 shutdown_connections 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faed00a2ec0 0x7faed00a5380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee81069c0 0x7faee819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee8101350 0x7faee819a950 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 --2- 172.21.15.46:0/4270100841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faee81009b0 0x7faee819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 >> 172.21.15.46:0/4270100841 conn(0x7faee8077990 msgr2=0x7faee80698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 shutdown_connections 2024-09-12T02:51:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:31.857+0000 7faeed047640 1 -- 172.21.15.46:0/4270100841 wait complete. 2024-09-12T02:51:31.871 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:31 smithi145 ceph-mon[34674]: pgmap v43: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:31.985 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:31 smithi196 ceph-mon[34587]: pgmap v43: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:32.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:31 smithi046 ceph-mon[30951]: pgmap v43: 129 pgs: 129 active+clean; 6.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 580 KiB/s wr, 231 op/s 2024-09-12T02:51:33.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:33 smithi145 ceph-mon[34674]: pgmap v44: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 11 MiB/s wr, 245 op/s 2024-09-12T02:51:33.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:33 smithi196 ceph-mon[34587]: pgmap v44: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 11 MiB/s wr, 245 op/s 2024-09-12T02:51:34.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:33 smithi046 ceph-mon[30951]: pgmap v44: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 11 MiB/s wr, 245 op/s 2024-09-12T02:51:35.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:35 smithi145 ceph-mon[34674]: pgmap v45: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 10 MiB/s wr, 74 op/s 2024-09-12T02:51:35.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:35 smithi196 ceph-mon[34587]: pgmap v45: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 10 MiB/s wr, 74 op/s 2024-09-12T02:51:36.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:35 smithi046 ceph-mon[30951]: pgmap v45: 129 pgs: 129 active+clean; 127 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 10 MiB/s wr, 74 op/s 2024-09-12T02:51:37.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:37 smithi145 ceph-mon[34674]: pgmap v46: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 21 MiB/s wr, 100 op/s 2024-09-12T02:51:37.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:37 smithi196 ceph-mon[34587]: pgmap v46: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 21 MiB/s wr, 100 op/s 2024-09-12T02:51:38.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:37 smithi046 ceph-mon[30951]: pgmap v46: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 21 MiB/s wr, 100 op/s 2024-09-12T02:51:39.834 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:39 smithi046 ceph-mon[30951]: pgmap v47: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:39 smithi145 ceph-mon[34674]: pgmap v47: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:39.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:39 smithi196 ceph-mon[34587]: pgmap v47: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:41 smithi145 ceph-mon[34674]: pgmap v48: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:41.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:41 smithi196 ceph-mon[34587]: pgmap v48: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:42.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:41 smithi046 ceph-mon[30951]: pgmap v48: 129 pgs: 129 active+clean; 259 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 39 op/s 2024-09-12T02:51:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:43 smithi145 ceph-mon[34674]: pgmap v49: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-09-12T02:51:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:43 smithi145 ceph-mon[34674]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-12T02:51:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:43 smithi196 ceph-mon[34587]: pgmap v49: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-09-12T02:51:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:43 smithi196 ceph-mon[34587]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-12T02:51:44.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:43 smithi046 ceph-mon[30951]: pgmap v49: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-09-12T02:51:44.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:43 smithi046 ceph-mon[30951]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 -- 172.21.15.46:0/3453540376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5881035a0 msgr2=0x7fe58810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/3453540376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5881035a0 0x7fe58810ba80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009900 tx=0x7fe57c02a480 comp rx=0 tx=0).stop 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 -- 172.21.15.46:0/3453540376 shutdown_connections 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/3453540376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe58810c0d0 0x7fe58810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/3453540376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5881035a0 0x7fe58810ba80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/3453540376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe588102c80 0x7fe588103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 -- 172.21.15.46:0/3453540376 >> 172.21.15.46:0/3453540376 conn(0x7fe588077990 msgr2=0x7fe5880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 -- 172.21.15.46:0/3453540376 shutdown_connections 2024-09-12T02:51:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.963+0000 7fe58d2a4640 1 -- 172.21.15.46:0/3453540376 wait complete. 2024-09-12T02:51:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 Processor -- start 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 -- start start 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe588102c80 0x7fe5881981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5881035a0 0x7fe588198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe588111d60 con 0x7fe58810c0d0 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe588111be0 con 0x7fe5881035a0 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.964+0000 7fe58d2a4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe588111ee0 con 0x7fe588102c80 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe586ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe588102c80 0x7fe5881981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52440/0 (socket says 172.21.15.46:52440) 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 -- 172.21.15.46:0/2611985948 learned_addr learned my addr 172.21.15.46:0/2611985948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5867fc640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5881035a0 0x7fe588198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 -- 172.21.15.46:0/2611985948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe588102c80 msgr2=0x7fe5881981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe588102c80 0x7fe5881981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 -- 172.21.15.46:0/2611985948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5881035a0 msgr2=0x7fe588198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5881035a0 0x7fe588198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 -- 172.21.15.46:0/2611985948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58819d1c0 con 0x7fe58810c0d0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe5877fe640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eaa0 tx=0x7fe58000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe56ffff640 1 -- 172.21.15.46:0/2611985948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004d20 con 0x7fe58810c0d0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe56ffff640 1 -- 172.21.15.46:0/2611985948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe580005020 con 0x7fe58810c0d0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58819d4b0 con 0x7fe58810c0d0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe56ffff640 1 -- 172.21.15.46:0/2611985948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5800116b0 con 0x7fe58810c0d0 2024-09-12T02:51:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.965+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5881a4d40 con 0x7fe58810c0d0 2024-09-12T02:51:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.968+0000 7fe56ffff640 1 -- 172.21.15.46:0/2611985948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5800051d0 con 0x7fe58810c0d0 2024-09-12T02:51:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.968+0000 7fe590036740 1 Processor -- start 2024-09-12T02:51:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.968+0000 7fe590036740 1 -- start start 2024-09-12T02:51:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 0x5579f5ddfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5de00c0 0x5579f5de24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5579f5d352f0 con 0x5579f5dd09f0 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5579f5cdd270 con 0x5579f5dd1310 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe590036740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5579f5d312b0 con 0x5579f5de00c0 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe56ffff640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 0x7fe5700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52452/0 (socket says 172.21.15.46:52452) 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 -- 172.21.15.46:0/1329110929 learned_addr learned my addr 172.21.15.46:0/1329110929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe586ffd640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 0x5579f5ddfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5877fe640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 0x7fe5700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe56ffff640 1 -- 172.21.15.46:0/2611985948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fe5800cf920 con 0x7fe58810c0d0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5de00c0 0x5579f5de24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 -- 172.21.15.46:0/1329110929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5de00c0 msgr2=0x5579f5de24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5de00c0 0x5579f5de24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 -- 172.21.15.46:0/1329110929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 msgr2=0x5579f5ddfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 0x5579f5ddfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe5867fc640 1 -- 172.21.15.46:0/1329110929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f5cdc340 con 0x5579f5dd09f0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.969+0000 7fe586ffd640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 0x5579f5ddfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.970+0000 7fe5877fe640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 0x7fe5700a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5800cd310 tx=0x7fe580090f70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.970+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005ab0 tx=0x7fe57c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b28dcd7830f61d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.970+0000 7fe56d7fa640 1 -- 172.21.15.46:0/1329110929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c035050 con 0x5579f5dd09f0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.970+0000 7fe56d7fa640 1 -- 172.21.15.46:0/1329110929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe57c031620 con 0x5579f5dd09f0 2024-09-12T02:51:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.970+0000 7fe56d7fa640 1 -- 172.21.15.46:0/1329110929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0055b0 con 0x5579f5dd09f0 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- 172.21.15.46:0/1329110929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 msgr2=0x5579f5dd0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005ab0 tx=0x7fe57c0058b0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- 172.21.15.46:0/1329110929 shutdown_connections 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5de00c0 0x5579f5de24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd1310 0x5579f5ddfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- 172.21.15.46:0/1329110929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5dd0dd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- 172.21.15.46:0/1329110929 >> 172.21.15.46:0/1329110929 conn(0x5579f5dcd060 msgr2=0x5579f5dcd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- 172.21.15.46:0/1329110929 shutdown_connections 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- 172.21.15.46:0/1329110929 wait complete. 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- start start 2024-09-12T02:51:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 Processor -- start 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd09f0 0x5579f5ddcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5ddd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5579f5de2cc0 con 0x5579f5de00c0 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5579f5cbe290 con 0x5579f5dd09f0 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.971+0000 7fe590036740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5579f5de2e40 con 0x5579f5dd1310 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe590036740 10 client.0 _ll_register_callbacks cb 0x5579f5cef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52466/0 (socket says 172.21.15.46:52466) 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 -- 172.21.15.46:0/4261422773 learned_addr learned my addr 172.21.15.46:0/4261422773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd09f0 0x5579f5ddcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe586ffd640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5ddd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 -- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 msgr2=0x5579f5ddd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5ddd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 -- 172.21.15.46:0/4261422773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd09f0 msgr2=0x5579f5ddcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd09f0 0x5579f5ddcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 -- 172.21.15.46:0/4261422773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f5a48210 con 0x5579f5de00c0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe5867fc640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0057c0 tx=0x7fe57c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad72cd96e26671c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe56aff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c051060 con 0x5579f5de00c0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fe57c0046c0 con 0x5579f5de00c0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c031620 con 0x5579f5de00c0 2024-09-12T02:51:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.972+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe57c035070 con 0x5579f5de00c0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 10 client.15453 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579f5b14620 con 0x5579f5de00c0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe57c031620 con 0x5579f5de00c0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe57c03aca0 con 0x5579f5de00c0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 10 client.15453 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 20 client.15453 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5defef0 0x5579f5df22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 4 client.15453 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579f5d42a20 con 0x5579f5defef0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.973+0000 7fe586ffd640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5defef0 0x5579f5df22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.974+0000 7fe56aff5640 10 client.15453 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.974+0000 7fe586ffd640 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5defef0 0x5579f5df22e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06f81864f86aaac server_cookie=acf7ed99de433b21 in_seq=0 out_seq=0 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe56aff5640 1 -- 172.21.15.46:0/4261422773 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579f5d42a20 con 0x5579f5defef0 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe56aff5640 10 client.15453 handle_command_reply: tid=0 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 client.15453 shutdown 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 msgr2=0x5579f5ddda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0057c0 tx=0x7fe57c03f040 comp rx=0 tx=0).stop 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 shutdown_connections 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5defef0 0x5579f5df22e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5de00c0 0x5579f5ddda20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5ddd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 --2- 172.21.15.46:0/4261422773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5dd09f0 0x5579f5ddcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 >> 172.21.15.46:0/4261422773 conn(0x5579f5dcd060 msgr2=0x5579f5dd2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe56aff5640 0 client.15453 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 shutdown_connections 2024-09-12T02:51:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 1 -- 172.21.15.46:0/4261422773 wait complete. 2024-09-12T02:51:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.975+0000 7fe590036740 20 client.15453 trim_cache size 0 max 16384 2024-09-12T02:51:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 Processor -- start 2024-09-12T02:51:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 -- start start 2024-09-12T02:51:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd09f0 0x5579f5df7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5dde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881af370 con 0x5579f5dd1310 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881acc40 con 0x5579f5de00c0 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe590036740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881aaa90 con 0x5579f5dd09f0 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52478/0 (socket says 172.21.15.46:52478) 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 -- 172.21.15.46:0/684877280 learned_addr learned my addr 172.21.15.46:0/684877280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe5867fc640 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd09f0 0x5579f5df7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe5867fc640 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5dde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 -- 172.21.15.46:0/684877280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd09f0 msgr2=0x5579f5df7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd09f0 0x5579f5df7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 -- 172.21.15.46:0/684877280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 msgr2=0x5579f5dde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5dde800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 -- 172.21.15.46:0/684877280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f5dcce90 con 0x5579f5dd1310 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.992+0000 7fe586ffd640 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5579f5dd36c0 tx=0x7fe57800acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44568dce0b96b43e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe56cff9640 1 -- 172.21.15.46:0/684877280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578005220 con 0x5579f5dd1310 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe56cff9640 1 -- 172.21.15.46:0/684877280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe578004c50 con 0x5579f5dd1310 2024-09-12T02:51:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe56cff9640 1 -- 172.21.15.46:0/684877280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57801d930 con 0x5579f5dd1310 2024-09-12T02:51:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 -- 172.21.15.46:0/684877280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 msgr2=0x5579f5df7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5579f5dd36c0 tx=0x7fe57800acf0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 -- 172.21.15.46:0/684877280 shutdown_connections 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5dde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd1310 0x5579f5df7dc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 --2- 172.21.15.46:0/684877280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd09f0 0x5579f5df7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 -- 172.21.15.46:0/684877280 >> 172.21.15.46:0/684877280 conn(0x5579f5dcd060 msgr2=0x5579f5dcd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.993+0000 7fe590036740 1 -- 172.21.15.46:0/684877280 shutdown_connections 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 -- 172.21.15.46:0/684877280 wait complete. 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 -- start start 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 Processor -- start 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5de87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5de3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881ab100 con 0x5579f5dd09f0 2024-09-12T02:51:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881ac570 con 0x5579f5de00c0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881ab7d0 con 0x5579f5dd1310 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe590036740 10 client.0 _ll_register_callbacks cb 0x7fe57800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe5867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52488/0 (socket says 172.21.15.46:52488) 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe5867fc640 1 -- 172.21.15.46:0/1497938406 learned_addr learned my addr 172.21.15.46:0/1497938406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5de3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.994+0000 7fe586ffd640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5de87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 -- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 msgr2=0x5579f5de87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5de87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 -- 172.21.15.46:0/1497938406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 msgr2=0x5579f5de3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5de3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 -- 172.21.15.46:0/1497938406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f5dd7d50 con 0x5579f5dd09f0 2024-09-12T02:51:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe5867fc640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005aa0 tx=0x7fe57c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306305e928a6a548 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe56aff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c051040 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fe57c0046c0 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe578004c50 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe57c009780 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe590036740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579f5e5a140 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.995+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe578004c50 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe57c03a710 con 0x5579f5dd09f0 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 20 client.15459 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5e5a6a0 0x5579f5e5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 4 client.15459 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:51:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579f5d3cd70 con 0x5579f5e5a6a0 2024-09-12T02:51:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe586ffd640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5e5a6a0 0x5579f5e5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe56aff5640 10 client.15459 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:44.996+0000 7fe586ffd640 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5e5a6a0 0x5579f5e5cae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f8efb1f09e0bc1 server_cookie=9072b928f5ebe090 in_seq=0 out_seq=0 2024-09-12T02:51:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe56aff5640 1 -- 172.21.15.46:0/1497938406 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579f5d3cd70 con 0x5579f5e5a6a0 2024-09-12T02:51:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe56aff5640 10 client.15459 handle_command_reply: tid=0 2024-09-12T02:51:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 client.15459 shutdown 2024-09-12T02:51:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 msgr2=0x5579f5de8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005aa0 tx=0x7fe57c03f040 comp rx=0 tx=0).stop 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 shutdown_connections 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5579f5e5a6a0 0x5579f5e5cae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.008+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5579f5de00c0 0x5579f5de3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579f5dd1310 0x5579f5de87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe56aff5640 0 client.15459 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 1 --2- 172.21.15.46:0/1497938406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5579f5dd09f0 0x5579f5de8270 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 >> 172.21.15.46:0/1497938406 conn(0x5579f5dcd060 msgr2=0x5579f5dde800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 shutdown_connections 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 1 -- 172.21.15.46:0/1497938406 wait complete. 2024-09-12T02:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe590036740 20 client.15459 trim_cache size 0 max 16384 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 msgr2=0x7fe5700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 0x7fe5700a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5800cd310 tx=0x7fe580090f70 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 msgr2=0x7fe58819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.009+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eaa0 tx=0x7fe58000eea0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 shutdown_connections 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe5700a2da0 0x7fe5700a5260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe58810c0d0 0x7fe58819ca90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5881035a0 0x7fe588198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 --2- 172.21.15.46:0/2611985948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe588102c80 0x7fe5881981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 >> 172.21.15.46:0/2611985948 conn(0x7fe588077990 msgr2=0x7fe5880fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 shutdown_connections 2024-09-12T02:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:45.010+0000 7fe58d2a4640 1 -- 172.21.15.46:0/2611985948 wait complete. 2024-09-12T02:51:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:45 smithi145 ceph-mon[34674]: pgmap v50: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 34 op/s 2024-09-12T02:51:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:45 smithi196 ceph-mon[34587]: pgmap v50: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 34 op/s 2024-09-12T02:51:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:45 smithi046 ceph-mon[30951]: pgmap v50: 129 pgs: 129 active+clean; 391 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 34 op/s 2024-09-12T02:51:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:47 smithi145 ceph-mon[34674]: pgmap v51: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 44 op/s 2024-09-12T02:51:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:47 smithi196 ceph-mon[34587]: pgmap v51: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 44 op/s 2024-09-12T02:51:48.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:47 smithi046 ceph-mon[30951]: pgmap v51: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 44 op/s 2024-09-12T02:51:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:49 smithi145 ceph-mon[34674]: pgmap v52: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:49.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:49 smithi196 ceph-mon[34587]: pgmap v52: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:50.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:49 smithi046 ceph-mon[30951]: pgmap v52: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:52.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:52 smithi196 ceph-mon[34587]: pgmap v53: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:52.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:52 smithi046 ceph-mon[30951]: pgmap v53: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:52.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:52 smithi145 ceph-mon[34674]: pgmap v53: 129 pgs: 129 active+clean; 527 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:51:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:54 smithi196 ceph-mon[34587]: pgmap v54: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:51:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:54 smithi046 ceph-mon[30951]: pgmap v54: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:51:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:54 smithi145 ceph-mon[34674]: pgmap v54: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:51:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:56 smithi196 ceph-mon[34587]: pgmap v55: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:51:56.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:56 smithi046 ceph-mon[30951]: pgmap v55: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:51:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:56 smithi145 ceph-mon[34674]: pgmap v55: 129 pgs: 129 active+clean; 659 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 -- 172.21.15.46:0/12671715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39581009b0 msgr2=0x7f3958100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 --2- 172.21.15.46:0/12671715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39581009b0 0x7f3958100e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f394c009930 tx=0x7f394c02a420 comp rx=0 tx=0).stop 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 -- 172.21.15.46:0/12671715 shutdown_connections 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 --2- 172.21.15.46:0/12671715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3958101350 0x7f395810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 --2- 172.21.15.46:0/12671715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39581009b0 0x7f3958100e10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 --2- 172.21.15.46:0/12671715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581069c0 0x7f3958106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.121+0000 7f395c8e6640 1 -- 172.21.15.46:0/12671715 >> 172.21.15.46:0/12671715 conn(0x7f3958077990 msgr2=0x7f39580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.122+0000 7f395c8e6640 1 -- 172.21.15.46:0/12671715 shutdown_connections 2024-09-12T02:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.122+0000 7f395c8e6640 1 -- 172.21.15.46:0/12671715 wait complete. 2024-09-12T02:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.122+0000 7f395c8e6640 1 Processor -- start 2024-09-12T02:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.122+0000 7f395c8e6640 1 -- start start 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581009b0 0x7f3958199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39581069c0 0x7f39581a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3958110cc0 con 0x7f3958101350 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3958110b40 con 0x7f39581069c0 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395c8e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3958110e40 con 0x7f39581009b0 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52706/0 (socket says 172.21.15.46:52706) 2024-09-12T02:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 -- 172.21.15.46:0/3831385693 learned_addr learned my addr 172.21.15.46:0/3831385693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f395659c640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581009b0 0x7f3958199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3956d9d640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39581069c0 0x7f39581a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 -- 172.21.15.46:0/3831385693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581009b0 msgr2=0x7f3958199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581009b0 0x7f3958199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 -- 172.21.15.46:0/3831385693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39581069c0 msgr2=0x7f39581a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39581069c0 0x7f39581a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 -- 172.21.15.46:0/3831385693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39581a4620 con 0x7f3958101350 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.123+0000 7f3955d9b640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f394c005ab0 tx=0x7f394c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.124+0000 7f39477fe640 1 -- 172.21.15.46:0/3831385693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c03f070 con 0x7f3958101350 2024-09-12T02:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.124+0000 7f39477fe640 1 -- 172.21.15.46:0/3831385693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f394c031b50 con 0x7f3958101350 2024-09-12T02:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.124+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581a47f0 con 0x7f3958101350 2024-09-12T02:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.124+0000 7f39477fe640 1 -- 172.21.15.46:0/3831385693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c035030 con 0x7f3958101350 2024-09-12T02:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.124+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39581a4c50 con 0x7f3958101350 2024-09-12T02:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.126+0000 7f39477fe640 1 -- 172.21.15.46:0/3831385693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f394c042070 con 0x7f3958101350 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f39477fe640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 0x7f39400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 Processor -- start 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f39477fe640 1 -- 172.21.15.46:0/3831385693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f394c0efe20 con 0x7f3958101350 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395659c640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 0x7f39400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 -- start start 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 0x565556cf4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf5410 0x565556cff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x565556c592f0 con 0x565556cffcb0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x565556c01270 con 0x565556cf5410 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.127+0000 7f395f67e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565556c552b0 con 0x565556cf4af0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f395659c640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 0x7f39400a5260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f394800ab60 tx=0x7f394800a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52716/0 (socket says 172.21.15.46:52716) 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 -- 172.21.15.46:0/2692546337 learned_addr learned my addr 172.21.15.46:0/2692546337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3955d9b640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf5410 0x565556cff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f395659c640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 0x565556cf4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 -- 172.21.15.46:0/2692546337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 msgr2=0x565556cf4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 0x565556cf4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 -- 172.21.15.46:0/2692546337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf5410 msgr2=0x565556cff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf5410 0x565556cff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 -- 172.21.15.46:0/2692546337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565556c00340 con 0x565556cffcb0 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f395659c640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 0x565556cf4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3956d9d640 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3950009fd0 tx=0x7f39500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e359edbabcced6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3944ff9640 1 -- 172.21.15.46:0/2692546337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950004d20 con 0x565556cffcb0 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3944ff9640 1 -- 172.21.15.46:0/2692546337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3950004050 con 0x565556cffcb0 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.128+0000 7f3944ff9640 1 -- 172.21.15.46:0/2692546337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950005280 con 0x565556cffcb0 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 -- 172.21.15.46:0/2692546337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 msgr2=0x565556d020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3950009fd0 tx=0x7f39500118d0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 -- 172.21.15.46:0/2692546337 shutdown_connections 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556d020a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf5410 0x565556cff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 --2- 172.21.15.46:0/2692546337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf4af0 0x565556cf4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 -- 172.21.15.46:0/2692546337 >> 172.21.15.46:0/2692546337 conn(0x565556cf1060 msgr2=0x565556cf1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 -- 172.21.15.46:0/2692546337 shutdown_connections 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.129+0000 7f395f67e740 1 -- 172.21.15.46:0/2692546337 wait complete. 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 -- start start 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 Processor -- start 2024-09-12T02:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556cfd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x565556d02880 con 0x565556cf5410 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x565556be2290 con 0x565556cf4af0 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565556d02a00 con 0x565556cffcb0 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395f67e740 10 client.0 _ll_register_callbacks cb 0x565556c13140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395659c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f3955d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395659c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:36548/0 (socket says 172.21.15.46:36548) 2024-09-12T02:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f395659c640 1 -- 172.21.15.46:0/1965950436 learned_addr learned my addr 172.21.15.46:0/1965950436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f3956d9d640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556cfd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.130+0000 7f3955d9b640 1 -- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 msgr2=0x565556cfd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f3955d9b640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556cfd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f3955d9b640 1 -- 172.21.15.46:0/1965950436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 msgr2=0x565556cfcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f3955d9b640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f3955d9b640 1 -- 172.21.15.46:0/1965950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555696c210 con 0x565556cf5410 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f3955d9b640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfd090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f394c0ed210 tx=0x7f394c0bbbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89de4ec59c942e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c049040 con 0x565556cf5410 2024-09-12T02:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f394c0ed490 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3950004050 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f394c09f050 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f395f67e740 10 client.15471 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565556a38620 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3950004050 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.131+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f394c0c47c0 con 0x565556cf5410 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 10 client.15471 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 20 client.15471 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d0fcb0 0x565556d12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 4 client.15471 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565556c66a20 con 0x565556d0fcb0 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395659c640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d0fcb0 0x565556d12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f395659c640 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d0fcb0 0x565556d12130 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb973c561d9a821e server_cookie=819daeaa27f7cba3 in_seq=0 out_seq=0 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.132+0000 7f393dffb640 10 client.15471 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f393dffb640 1 -- 172.21.15.46:0/1965950436 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565556c66a20 con 0x565556d0fcb0 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f393dffb640 10 client.15471 handle_command_reply: tid=0 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 client.15471 shutdown 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 msgr2=0x565556cfd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfd090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f394c0ed210 tx=0x7f394c0bbbe0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 shutdown_connections 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d0fcb0 0x565556d12130 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556cfd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfd090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 --2- 172.21.15.46:0/1965950436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 >> 172.21.15.46:0/1965950436 conn(0x565556cf1060 msgr2=0x565556cf6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f393dffb640 0 client.15471 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 shutdown_connections 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 1 -- 172.21.15.46:0/1965950436 wait complete. 2024-09-12T02:51:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.133+0000 7f395f67e740 20 client.15471 trim_cache size 0 max 16384 2024-09-12T02:51:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 Processor -- start 2024-09-12T02:51:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 -- start start 2024-09-12T02:51:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556d17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39581b1550 con 0x565556cf5410 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39581aee20 con 0x565556cf4af0 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395f67e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39581acc70 con 0x565556cffcb0 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f3955d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f3955d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52720/0 (socket says 172.21.15.46:52720) 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f3955d9b640 1 -- 172.21.15.46:0/2616125293 learned_addr learned my addr 172.21.15.46:0/2616125293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f3956d9d640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556d17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 -- 172.21.15.46:0/2616125293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 msgr2=0x565556d17a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556d17a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 -- 172.21.15.46:0/2616125293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 msgr2=0x565556cfeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f395659c640 1 -- 172.21.15.46:0/2616125293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565556cf5240 con 0x565556cf4af0 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.154+0000 7f3955d9b640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.155+0000 7f395659c640 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfd400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f394800f850 tx=0x7f3948012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90be589cbe605466 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.155+0000 7f393ffff640 1 -- 172.21.15.46:0/2616125293 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948016070 con 0x565556cf4af0 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.155+0000 7f393ffff640 1 -- 172.21.15.46:0/2616125293 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3948027860 con 0x565556cf4af0 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.155+0000 7f393ffff640 1 -- 172.21.15.46:0/2616125293 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480268c0 con 0x565556cf4af0 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 -- 172.21.15.46:0/2616125293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 msgr2=0x565556cfd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfd400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f394800f850 tx=0x7f3948012000 comp rx=0 tx=0).stop 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 -- 172.21.15.46:0/2616125293 shutdown_connections 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cffcb0 0x565556d17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cf5410 0x565556cfeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 --2- 172.21.15.46:0/2616125293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556cfd400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 -- 172.21.15.46:0/2616125293 >> 172.21.15.46:0/2616125293 conn(0x565556cf1060 msgr2=0x565556cfe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 -- 172.21.15.46:0/2616125293 shutdown_connections 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.156+0000 7f395f67e740 1 -- 172.21.15.46:0/2616125293 wait complete. 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 -- start start 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 Processor -- start 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556d07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf5410 0x565556d081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ad2e0 con 0x565556cffcb0 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ae750 con 0x565556cf4af0 2024-09-12T02:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ad9b0 con 0x565556cf5410 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f3956d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f3956d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52724/0 (socket says 172.21.15.46:52724) 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395f67e740 10 client.0 _ll_register_callbacks cb 0x7f394c0b8280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f3956d9d640 1 -- 172.21.15.46:0/3453544901 learned_addr learned my addr 172.21.15.46:0/3453544901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f3955d9b640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf5410 0x565556d081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.157+0000 7f395659c640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556d07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 -- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf5410 msgr2=0x565556d081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf5410 0x565556d081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 -- 172.21.15.46:0/3453544901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 msgr2=0x565556d07c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556d07c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 -- 172.21.15.46:0/3453544901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565556cf94c0 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f3956d9d640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3950014040 tx=0x7f3950009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18c7d51b6c0b7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f393dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950004d10 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f3950004050 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3948027860 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3950035050 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f395f67e740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.158+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565556d7a1c0 con 0x565556cffcb0 2024-09-12T02:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3948027860 con 0x565556cffcb0 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f39500054f0 con 0x565556cffcb0 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 20 client.15477 populate_metadata read hostname 'smithi046' 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d7a830 0x565556d7cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 4 client.15477 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.159+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565556c60d70 con 0x565556d7a830 2024-09-12T02:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.160+0000 7f395659c640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d7a830 0x565556d7cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.162+0000 7f393dffb640 10 client.15477 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.162+0000 7f395659c640 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d7a830 0x565556d7cc50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cfeec1b34836661 server_cookie=dffc6484e66893b0 in_seq=0 out_seq=0 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f393dffb640 1 -- 172.21.15.46:0/3453544901 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565556c60d70 con 0x565556d7a830 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f393dffb640 10 client.15477 handle_command_reply: tid=0 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 client.15477 shutdown 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 msgr2=0x565556cf8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3950014040 tx=0x7f3950009d60 comp rx=0 tx=0).stop 2024-09-12T02:51:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 shutdown_connections 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x565556d7a830 0x565556d7cc50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565556cffcb0 0x565556cf8f80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565556cf5410 0x565556d081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f393dffb640 0 client.15477 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 --2- 172.21.15.46:0/3453544901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565556cf4af0 0x565556d07c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 >> 172.21.15.46:0/3453544901 conn(0x565556cf1060 msgr2=0x565556cf01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 shutdown_connections 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 1 -- 172.21.15.46:0/3453544901 wait complete. 2024-09-12T02:51:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.171+0000 7f395f67e740 20 client.15477 trim_cache size 0 max 16384 2024-09-12T02:51:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:51:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:51:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:51:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 msgr2=0x7f39400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 0x7f39400a5260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f394800ab60 tx=0x7f394800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 msgr2=0x7f39581995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f394c005ab0 tx=0x7f394c003b90 comp rx=0 tx=0).stop 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 shutdown_connections 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f39400a2da0 0x7f39400a5260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39581069c0 0x7f39581a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3958101350 0x7f39581995a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 --2- 172.21.15.46:0/3831385693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39581009b0 0x7f3958199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 >> 172.21.15.46:0/3831385693 conn(0x7f3958077990 msgr2=0x7f3958103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 shutdown_connections 2024-09-12T02:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:51:58.172+0000 7f395c8e6640 1 -- 172.21.15.46:0/3831385693 wait complete. 2024-09-12T02:51:58.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:51:58 smithi196 ceph-mon[34587]: pgmap v56: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:51:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:51:58 smithi046 ceph-mon[30951]: pgmap v56: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:51:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:51:58 smithi145 ceph-mon[34674]: pgmap v56: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:00.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:00 smithi196 ceph-mon[34587]: pgmap v57: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:00 smithi046 ceph-mon[30951]: pgmap v57: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:00 smithi145 ceph-mon[34674]: pgmap v57: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:02 smithi196 ceph-mon[34587]: pgmap v58: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:02 smithi046 ceph-mon[30951]: pgmap v58: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:02 smithi145 ceph-mon[34674]: pgmap v58: 129 pgs: 129 active+clean; 791 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:04 smithi196 ceph-mon[34587]: pgmap v59: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:04 smithi046 ceph-mon[30951]: pgmap v59: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:04 smithi145 ceph-mon[34674]: pgmap v59: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:06.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:06 smithi196 ceph-mon[34587]: pgmap v60: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:06 smithi046 ceph-mon[30951]: pgmap v60: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:06 smithi145 ceph-mon[34674]: pgmap v60: 129 pgs: 129 active+clean; 923 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:08.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:08 smithi196 ceph-mon[34587]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:08.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:08 smithi046 ceph-mon[30951]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:08 smithi145 ceph-mon[34674]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:52:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:10 smithi196 ceph-mon[34587]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:10 smithi046 ceph-mon[30951]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:10 smithi145 ceph-mon[34674]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 -- 172.21.15.46:0/2427597024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141009b0 msgr2=0x7fd114100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2427597024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141009b0 0x7fd114100e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd108009930 tx=0x7fd10802a420 comp rx=0 tx=0).stop 2024-09-12T02:52:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 -- 172.21.15.46:0/2427597024 shutdown_connections 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2427597024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 0x7fd11410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2427597024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141009b0 0x7fd114100e10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2427597024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141069c0 0x7fd114106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.277+0000 7fd119b6d640 1 -- 172.21.15.46:0/2427597024 >> 172.21.15.46:0/2427597024 conn(0x7fd114077990 msgr2=0x7fd1140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.278+0000 7fd119b6d640 1 -- 172.21.15.46:0/2427597024 shutdown_connections 2024-09-12T02:52:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.278+0000 7fd119b6d640 1 -- 172.21.15.46:0/2427597024 wait complete. 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.278+0000 7fd119b6d640 1 Processor -- start 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.278+0000 7fd119b6d640 1 -- start start 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141009b0 0x7fd1141982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 0x7fd114198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd114110cb0 con 0x7fd1141069c0 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd114110b30 con 0x7fd1141009b0 2024-09-12T02:52:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd119b6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd114110e30 con 0x7fd114101350 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47348/0 (socket says 172.21.15.46:47348) 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 -- 172.21.15.46:0/2507327319 learned_addr learned my addr 172.21.15.46:0/2507327319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd112ffd640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 0x7fd114198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd1137fe640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141009b0 0x7fd1141982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 -- 172.21.15.46:0/2507327319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 msgr2=0x7fd114198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 0x7fd114198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 -- 172.21.15.46:0/2507327319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141009b0 msgr2=0x7fd1141982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141009b0 0x7fd1141982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 -- 172.21.15.46:0/2507327319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11419d2b0 con 0x7fd1141069c0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.279+0000 7fd113fff640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00eb10 tx=0x7fd10c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.280+0000 7fd110ff9640 1 -- 172.21.15.46:0/2507327319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c004b20 con 0x7fd1141069c0 2024-09-12T02:52:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.280+0000 7fd110ff9640 1 -- 172.21.15.46:0/2507327319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd10c004cc0 con 0x7fd1141069c0 2024-09-12T02:52:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.280+0000 7fd110ff9640 1 -- 172.21.15.46:0/2507327319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c010690 con 0x7fd1141069c0 2024-09-12T02:52:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.280+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11419d5a0 con 0x7fd1141069c0 2024-09-12T02:52:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.280+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141a4e30 con 0x7fd1141069c0 2024-09-12T02:52:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.282+0000 7fd110ff9640 1 -- 172.21.15.46:0/2507327319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd10c005020 con 0x7fd1141069c0 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 Processor -- start 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 -- start start 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a1c02c2f0 con 0x559a1c0c82f0 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a1bfd4270 con 0x559a1c0d70c0 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a1c0282b0 con 0x559a1c0c79d0 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd1137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd1137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47358/0 (socket says 172.21.15.46:47358) 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd1137fe640 1 -- 172.21.15.46:0/454657743 learned_addr learned my addr 172.21.15.46:0/454657743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd110ff9640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 0x7fd0fc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd112ffd640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd112ffd640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.283+0000 7fd113fff640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 0x7fd0fc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 -- 172.21.15.46:0/454657743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 msgr2=0x559a1c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 -- 172.21.15.46:0/454657743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 msgr2=0x559a1c0d94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 -- 172.21.15.46:0/454657743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1bfd3340 con 0x559a1c0c82f0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd112ffd640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd110ff9640 1 -- 172.21.15.46:0/2507327319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fd10c0cffc0 con 0x7fd1141069c0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd1137fe640 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd10400dc60 tx=0x7fd10400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99dd2520a699ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd0f9ffb640 1 -- 172.21.15.46:0/454657743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104011440 con 0x559a1c0c82f0 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd113fff640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 0x7fd0fc0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0cd460 tx=0x7fd10c098500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd0f9ffb640 1 -- 172.21.15.46:0/454657743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd104017070 con 0x559a1c0c82f0 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.284+0000 7fd0f9ffb640 1 -- 172.21.15.46:0/454657743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104010860 con 0x559a1c0c82f0 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- 172.21.15.46:0/454657743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 msgr2=0x559a1c0d6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd10400dc60 tx=0x7fd10400f730 comp rx=0 tx=0).stop 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- 172.21.15.46:0/454657743 shutdown_connections 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c82f0 0x559a1c0d6b80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/454657743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- 172.21.15.46:0/454657743 >> 172.21.15.46:0/454657743 conn(0x559a1c0c4060 msgr2=0x559a1c0c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- 172.21.15.46:0/454657743 shutdown_connections 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- 172.21.15.46:0/454657743 wait complete. 2024-09-12T02:52:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.285+0000 7fd11c9e5740 1 -- start start 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 Processor -- start 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a1c0d9cf0 con 0x559a1c0c79d0 2024-09-12T02:52:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a1bfb5290 con 0x559a1c0d70c0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a1c0d9e70 con 0x559a1c0c82f0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd11c9e5740 10 client.0 _ll_register_callbacks cb 0x559a1bfe6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47362/0 (socket says 172.21.15.46:47362) 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 -- 172.21.15.46:0/1882851332 learned_addr learned my addr 172.21.15.46:0/1882851332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd1137fe640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 -- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 msgr2=0x559a1c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 -- 172.21.15.46:0/1882851332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 msgr2=0x559a1c0d4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.286+0000 7fd112ffd640 1 -- 172.21.15.46:0/1882851332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1bd3f210 con 0x559a1c0c79d0 2024-09-12T02:52:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd112ffd640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd1080096c0 tx=0x7fd108009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff87287006ff84f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108035040 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fd108004450 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104017070 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd108050050 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 10 client.15489 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a1be0b620 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd104017070 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd10803a040 con 0x559a1c0c79d0 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 20 client.15489 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c0e7120 0x559a1c0e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 4 client.15489 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.287+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1c039a20 con 0x559a1c0e7120 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.288+0000 7fd1137fe640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c0e7120 0x559a1c0e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.288+0000 7fd0f2ffd640 10 client.15489 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.288+0000 7fd1137fe640 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c0e7120 0x559a1c0e9550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a491903c0ef063e server_cookie=7862a5adc74db207 in_seq=0 out_seq=0 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/1882851332 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a1c039a20 con 0x559a1c0e7120 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd0f2ffd640 10 client.15489 handle_command_reply: tid=0 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 client.15489 shutdown 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 msgr2=0x559a1c0d4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd1080096c0 tx=0x7fd108009260 comp rx=0 tx=0).stop 2024-09-12T02:52:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 shutdown_connections 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c0e7120 0x559a1c0e9550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0d4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd0f2ffd640 0 client.15489 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/1882851332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0d4060 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 >> 172.21.15.46:0/1882851332 conn(0x559a1c0c4060 msgr2=0x559a1c0c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 shutdown_connections 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 1 -- 172.21.15.46:0/1882851332 wait complete. 2024-09-12T02:52:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.289+0000 7fd11c9e5740 20 client.15489 trim_cache size 0 max 16384 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 Processor -- start 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 -- start start 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0c82f0 0x559a1c0eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141af550 con 0x559a1c0d70c0 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141acdf0 con 0x559a1c0c82f0 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.305+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141aac40 con 0x559a1c0c79d0 2024-09-12T02:52:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47368/0 (socket says 172.21.15.46:47368) 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 -- 172.21.15.46:0/2205739827 learned_addr learned my addr 172.21.15.46:0/2205739827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd1137fe640 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0c82f0 0x559a1c0eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 -- 172.21.15.46:0/2205739827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 msgr2=0x559a1c0e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 -- 172.21.15.46:0/2205739827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0c82f0 msgr2=0x559a1c0eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0c82f0 0x559a1c0eee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 -- 172.21.15.46:0/2205739827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1c0c8f50 con 0x559a1c0d70c0 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd112ffd640 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd108005ec0 tx=0x7fd108031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821e809538c56d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd0f97fa640 1 -- 172.21.15.46:0/2205739827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108035070 con 0x559a1c0d70c0 2024-09-12T02:52:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd0f97fa640 1 -- 172.21.15.46:0/2205739827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd108047070 con 0x559a1c0d70c0 2024-09-12T02:52:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.306+0000 7fd0f97fa640 1 -- 172.21.15.46:0/2205739827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108003a30 con 0x559a1c0d70c0 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- 172.21.15.46:0/2205739827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 msgr2=0x559a1c0ef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd108005ec0 tx=0x7fd108031400 comp rx=0 tx=0).stop 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- 172.21.15.46:0/2205739827 shutdown_connections 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0d70c0 0x559a1c0ef3a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0c82f0 0x559a1c0eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/2205739827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c79d0 0x559a1c0e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- 172.21.15.46:0/2205739827 >> 172.21.15.46:0/2205739827 conn(0x559a1c0c4060 msgr2=0x559a1c0d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- 172.21.15.46:0/2205739827 shutdown_connections 2024-09-12T02:52:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- 172.21.15.46:0/2205739827 wait complete. 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.307+0000 7fd11c9e5740 1 -- start start 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 Processor -- start 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ab2b0 con 0x559a1c0c79d0 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ac720 con 0x559a1c0d70c0 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ab980 con 0x559a1c0c82f0 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd11c9e5740 10 client.0 _ll_register_callbacks cb 0x7fd108009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd112ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47370/0 (socket says 172.21.15.46:47370) 2024-09-12T02:52:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd112ffd640 1 -- 172.21.15.46:0/745377202 learned_addr learned my addr 172.21.15.46:0/745377202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd112ffd640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.308+0000 7fd1137fe640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 -- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 msgr2=0x559a1c0db370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0db370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 -- 172.21.15.46:0/745377202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 msgr2=0x559a1c0db8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0db8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 -- 172.21.15.46:0/745377202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1c150e60 con 0x559a1c0c79d0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd112ffd640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd10803d6e0 tx=0x7fd1080035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b5ec81b2a0b3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108035070 con 0x559a1c0c79d0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd108047070 con 0x559a1c0c79d0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fd108031aa0 con 0x559a1c0c79d0 2024-09-12T02:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd10806b050 con 0x559a1c0c79d0 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd11c9e5740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a1c152130 con 0x559a1c0c79d0 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.309+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd108047070 con 0x559a1c0c79d0 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd108054400 con 0x559a1c0c79d0 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 20 client.15495 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c152850 0x559a1c154c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 4 client.15495 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a1c033d70 con 0x559a1c152850 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd1137fe640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c152850 0x559a1c154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd0f2ffd640 10 client.15495 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.310+0000 7fd1137fe640 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c152850 0x559a1c154c40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774aea6a519e04e5 server_cookie=f13447ab79ccc06 in_seq=0 out_seq=0 2024-09-12T02:52:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.318+0000 7fd0f2ffd640 1 -- 172.21.15.46:0/745377202 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a1c033d70 con 0x559a1c152850 2024-09-12T02:52:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.318+0000 7fd0f2ffd640 10 client.15495 handle_command_reply: tid=0 2024-09-12T02:52:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.318+0000 7fd11c9e5740 1 client.15495 shutdown 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.318+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 msgr2=0x559a1c0dae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.318+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd10803d6e0 tx=0x7fd1080035f0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 shutdown_connections 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a1c152850 0x559a1c154c40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a1c0d70c0 0x559a1c0db8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd0f2ffd640 0 client.15495 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a1c0c82f0 0x559a1c0db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 --2- 172.21.15.46:0/745377202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a1c0c79d0 0x559a1c0dae30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 >> 172.21.15.46:0/745377202 conn(0x559a1c0c4060 msgr2=0x559a1c0c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 shutdown_connections 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 1 -- 172.21.15.46:0/745377202 wait complete. 2024-09-12T02:52:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd11c9e5740 20 client.15495 trim_cache size 0 max 16384 2024-09-12T02:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 msgr2=0x7fd0fc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 0x7fd0fc0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0cd460 tx=0x7fd10c098500 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 msgr2=0x7fd11419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.319+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00eb10 tx=0x7fd10c00c2d0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 shutdown_connections 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd0fc0a2da0 0x7fd0fc0a5260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1141069c0 0x7fd11419cb80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd114101350 0x7fd114198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 --2- 172.21.15.46:0/2507327319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1141009b0 0x7fd1141982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 >> 172.21.15.46:0/2507327319 conn(0x7fd114077990 msgr2=0x7fd114069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 shutdown_connections 2024-09-12T02:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:11.320+0000 7fd119b6d640 1 -- 172.21.15.46:0/2507327319 wait complete. 2024-09-12T02:52:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:12 smithi196 ceph-mon[34587]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:12 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:52:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:12 smithi046 ceph-mon[30951]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:12 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:52:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:12 smithi145 ceph-mon[34674]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:52:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:12 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:52:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:14 smithi196 ceph-mon[34587]: pgmap v64: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 41 op/s 2024-09-12T02:52:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:14 smithi046 ceph-mon[30951]: pgmap v64: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 41 op/s 2024-09-12T02:52:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:14 smithi145 ceph-mon[34674]: pgmap v64: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 41 op/s 2024-09-12T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:16 smithi196 ceph-mon[34587]: pgmap v65: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:16 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:16 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:16.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:16 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:16.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:16 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:52:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:16 smithi046 ceph-mon[30951]: pgmap v65: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:16 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:16 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:16 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:16 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:52:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:16 smithi145 ceph-mon[34674]: pgmap v65: 129 pgs: 129 active+clean; 1.2 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:16 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:16 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:16 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:16 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:52:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:18 smithi196 ceph-mon[34587]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:18 smithi046 ceph-mon[30951]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:18 smithi145 ceph-mon[34674]: pgmap v66: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:20.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:20 smithi196 ceph-mon[34587]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:20 smithi046 ceph-mon[30951]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:20.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:20 smithi145 ceph-mon[34674]: pgmap v67: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:22.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:22 smithi196 ceph-mon[34587]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:22 smithi046 ceph-mon[30951]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:22 smithi145 ceph-mon[34674]: pgmap v68: 129 pgs: 129 active+clean; 1.3 GiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:52:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 -- 172.21.15.46:0/3916559445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a0102c80 msgr2=0x7f51a0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/3916559445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a0102c80 0x7f51a0103060 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5190009900 tx=0x7f519002a480 comp rx=0 tx=0).stop 2024-09-12T02:52:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 -- 172.21.15.46:0/3916559445 shutdown_connections 2024-09-12T02:52:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/3916559445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a010c0d0 0x7f51a010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/3916559445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a01035a0 0x7f51a010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/3916559445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a0102c80 0x7f51a0103060 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 -- 172.21.15.46:0/3916559445 >> 172.21.15.46:0/3916559445 conn(0x7f51a0077990 msgr2=0x7f51a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 -- 172.21.15.46:0/3916559445 shutdown_connections 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.419+0000 7f51a4e8b640 1 -- 172.21.15.46:0/3916559445 wait complete. 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 Processor -- start 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 -- start start 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a0102c80 0x7f51a0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a010c0d0 0x7f51a019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0111d60 con 0x7f51a01035a0 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0111be0 con 0x7f51a010c0d0 2024-09-12T02:52:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.420+0000 7f51a4e8b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0111ee0 con 0x7f51a0102c80 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38476/0 (socket says 172.21.15.46:38476) 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 -- 172.21.15.46:0/1463896998 learned_addr learned my addr 172.21.15.46:0/1463896998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519e59c640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a0102c80 0x7f51a0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a010c0d0 0x7f51a019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 -- 172.21.15.46:0/1463896998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a0102c80 msgr2=0x7f51a0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a0102c80 0x7f51a0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 -- 172.21.15.46:0/1463896998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a010c0d0 msgr2=0x7f51a019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a010c0d0 0x7f51a019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 -- 172.21.15.46:0/1463896998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a019d230 con 0x7f51a01035a0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f519400d950 tx=0x7f519400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f518f7fe640 1 -- 172.21.15.46:0/1463896998 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194017030 con 0x7f51a01035a0 2024-09-12T02:52:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f518f7fe640 1 -- 172.21.15.46:0/1463896998 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5194004020 con 0x7f51a01035a0 2024-09-12T02:52:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a019d520 con 0x7f51a01035a0 2024-09-12T02:52:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f518f7fe640 1 -- 172.21.15.46:0/1463896998 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51940055e0 con 0x7f51a01035a0 2024-09-12T02:52:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.421+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01a4db0 con 0x7f51a01035a0 2024-09-12T02:52:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.424+0000 7f518f7fe640 1 -- 172.21.15.46:0/1463896998 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51940041d0 con 0x7f51a01035a0 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 Processor -- start 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 -- start start 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f518f7fe640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 0x7f51880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a47007f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 0x7f51880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f518f7fe640 1 -- 172.21.15.46:0/1463896998 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f51940cf0a0 con 0x7f51a01035a0 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a46ffdf2f0 con 0x55a47007f820 2024-09-12T02:52:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a46ff87270 con 0x55a47007ef00 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f51a7c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a46ffdb2b0 con 0x55a47008a0c0 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38492/0 (socket says 172.21.15.46:38492) 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f519dd9b640 1 -- 172.21.15.46:0/3998636746 learned_addr learned my addr 172.21.15.46:0/3998636746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.425+0000 7f519e59c640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a47007f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519ed9d640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 -- 172.21.15.46:0/3998636746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 msgr2=0x55a47008c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 -- 172.21.15.46:0/3998636746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 msgr2=0x55a47007f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a47007f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 -- 172.21.15.46:0/3998636746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46ff86340 con 0x55a47007f820 2024-09-12T02:52:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519ed9d640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 0x7f51880a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5198005fd0 tx=0x7f51980107a0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f519dd9b640 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f519400dfd0 tx=0x7f519409aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5532074bba335df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f518cff9640 1 -- 172.21.15.46:0/3998636746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51940a3ed0 con 0x55a47007f820 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f518cff9640 1 -- 172.21.15.46:0/3998636746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51940cc710 con 0x55a47007f820 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.426+0000 7f518cff9640 1 -- 172.21.15.46:0/3998636746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519409f170 con 0x55a47007f820 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- 172.21.15.46:0/3998636746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 msgr2=0x55a470089b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f519400dfd0 tx=0x7f519409aeb0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- 172.21.15.46:0/3998636746 shutdown_connections 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007f820 0x55a470089b80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 --2- 172.21.15.46:0/3998636746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a47007f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- 172.21.15.46:0/3998636746 >> 172.21.15.46:0/3998636746 conn(0x55a47007b470 msgr2=0x55a47007b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- 172.21.15.46:0/3998636746 shutdown_connections 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- 172.21.15.46:0/3998636746 wait complete. 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.427+0000 7f51a7c23740 1 -- start start 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 Processor -- start 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007ef00 0x55a4700870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a470087600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a47008cc90 con 0x55a47008a0c0 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a46ff68290 con 0x55a47007f820 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a47008ce10 con 0x55a47007ef00 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f51a7c23740 10 client.0 _ll_register_callbacks cb 0x55a46ff99140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38502/0 (socket says 172.21.15.46:38502) 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 -- 172.21.15.46:0/1149791254 learned_addr learned my addr 172.21.15.46:0/1149791254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519e59c640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007ef00 0x55a4700870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a470087600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 -- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007ef00 msgr2=0x55a4700870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007ef00 0x55a4700870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 -- 172.21.15.46:0/1149791254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 msgr2=0x55a470087600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a470087600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519ed9d640 1 -- 172.21.15.46:0/1149791254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46fcf2210 con 0x55a47008a0c0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.428+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a470087600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5198011c40 tx=0x7f5198004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e740aabc70138f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f5185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198020620 con 0x55a47008a0c0 2024-09-12T02:52:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f51980207c0 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51940cc710 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f519802f050 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f51a7c23740 10 client.15507 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a46fdbe620 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.429+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51940cc710 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5198028990 con 0x55a47008a0c0 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 20 client.15507 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a47009a140 0x55a47009c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 4 client.15507 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46ffeca20 con 0x55a47009a140 2024-09-12T02:52:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f519e59c640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a47009a140 0x55a47009c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f519e59c640 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a47009a140 0x55a47009c550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ceccead8a69292 server_cookie=fd4fcacc8ddb5c3b in_seq=0 out_seq=0 2024-09-12T02:52:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.430+0000 7f5185ffb640 10 client.15507 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f5185ffb640 1 -- 172.21.15.46:0/1149791254 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a46ffeca20 con 0x55a47009a140 2024-09-12T02:52:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f5185ffb640 10 client.15507 handle_command_reply: tid=0 2024-09-12T02:52:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 client.15507 shutdown 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 msgr2=0x55a470087b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5198011c40 tx=0x7f5198004ab0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 shutdown_connections 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a47009a140 0x55a47009c550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a470087b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a470087600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f5185ffb640 0 client.15507 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1149791254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007ef00 0x55a4700870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 >> 172.21.15.46:0/1149791254 conn(0x55a47007b470 msgr2=0x55a4700761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 shutdown_connections 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 1 -- 172.21.15.46:0/1149791254 wait complete. 2024-09-12T02:52:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.431+0000 7f51a7c23740 20 client.15507 trim_cache size 0 max 16384 2024-09-12T02:52:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.445+0000 7f51a7c23740 1 Processor -- start 2024-09-12T02:52:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.445+0000 7f51a7c23740 1 -- start start 2024-09-12T02:52:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a4700819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 0x55a4700a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01af4d0 con 0x55a47008a0c0 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01acd70 con 0x55a47007ef00 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f51a7c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01aabc0 con 0x55a47007f820 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38506/0 (socket says 172.21.15.46:38506) 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 -- 172.21.15.46:0/1436420297 learned_addr learned my addr 172.21.15.46:0/1436420297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 0x55a4700a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519e59c640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a4700819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 -- 172.21.15.46:0/1436420297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 msgr2=0x55a4700a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 0x55a4700a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 -- 172.21.15.46:0/1436420297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 msgr2=0x55a4700819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a4700819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519ed9d640 1 -- 172.21.15.46:0/1436420297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47007c100 con 0x55a47008a0c0 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.446+0000 7f519dd9b640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 0x55a4700a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.447+0000 7f519ed9d640 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5198011c10 tx=0x7f5198010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71d23313770f7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.447+0000 7f5187fff640 1 -- 172.21.15.46:0/1436420297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198037070 con 0x55a47008a0c0 2024-09-12T02:52:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.447+0000 7f5187fff640 1 -- 172.21.15.46:0/1436420297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5198009c40 con 0x55a47008a0c0 2024-09-12T02:52:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.447+0000 7f5187fff640 1 -- 172.21.15.46:0/1436420297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198004040 con 0x55a47008a0c0 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- 172.21.15.46:0/1436420297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 msgr2=0x55a4700a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5198011c10 tx=0x7f5198010d70 comp rx=0 tx=0).stop 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- 172.21.15.46:0/1436420297 shutdown_connections 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47008a0c0 0x55a4700a23a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47007f820 0x55a4700a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- 172.21.15.46:0/1436420297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007ef00 0x55a4700819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- 172.21.15.46:0/1436420297 >> 172.21.15.46:0/1436420297 conn(0x55a47007b470 msgr2=0x55a470089140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- 172.21.15.46:0/1436420297 shutdown_connections 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- 172.21.15.46:0/1436420297 wait complete. 2024-09-12T02:52:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- start start 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 Processor -- start 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a4700925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01ab230 con 0x55a47007ef00 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01ac6a0 con 0x55a47007f820 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51a01ab900 con 0x55a47008a0c0 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.448+0000 7f51a7c23740 10 client.0 _ll_register_callbacks cb 0x7f5198009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38516/0 (socket says 172.21.15.46:38516) 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 -- 172.21.15.46:0/145567724 learned_addr learned my addr 172.21.15.46:0/145567724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519dd9b640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a4700925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519ed9d640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 -- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 msgr2=0x55a47008f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 -- 172.21.15.46:0/145567724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 msgr2=0x55a4700925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a4700925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 -- 172.21.15.46:0/145567724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a470092cd0 con 0x55a47007ef00 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f519e59c640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5190009850 tx=0x7f5190004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae0433c873cc8cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f5185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190035040 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f5190005040 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.449+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5198009c40 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5190055050 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 10 client.15513 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4701054b0 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5198009c40 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51900035f0 con 0x55a47007ef00 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 20 client.15513 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a470105c30 0x55a470108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 4 client.15513 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:52:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a46ffe6d70 con 0x55a470105c30 2024-09-12T02:52:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.450+0000 7f519dd9b640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a470105c30 0x55a470108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.451+0000 7f519dd9b640 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a470105c30 0x55a470108050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1858b4c0ee235d6a server_cookie=80c930ae2e136bf1 in_seq=0 out_seq=0 2024-09-12T02:52:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.451+0000 7f5185ffb640 10 client.15513 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f5185ffb640 1 -- 172.21.15.46:0/145567724 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a46ffe6d70 con 0x55a470105c30 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f5185ffb640 10 client.15513 handle_command_reply: tid=0 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 client.15513 shutdown 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 msgr2=0x55a4700920a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5190009850 tx=0x7f5190004040 comp rx=0 tx=0).stop 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 shutdown_connections 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a470105c30 0x55a470108050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a47008a0c0 0x55a47008f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a47007f820 0x55a4700925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 --2- 172.21.15.46:0/145567724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a47007ef00 0x55a4700920a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 >> 172.21.15.46:0/145567724 conn(0x55a47007b470 msgr2=0x55a4700759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f5185ffb640 0 client.15513 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 shutdown_connections 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 1 -- 172.21.15.46:0/145567724 wait complete. 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.458+0000 7f51a7c23740 20 client.15513 trim_cache size 0 max 16384 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:52:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:52:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 msgr2=0x7f51880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 0x7f51880a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5198005fd0 tx=0x7f51980107a0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 msgr2=0x7f51a0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f519400d950 tx=0x7f519400de10 comp rx=0 tx=0).stop 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 shutdown_connections 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51880a2da0 0x7f51880a5260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a010c0d0 0x7f51a019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a01035a0 0x7f51a0198780 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 --2- 172.21.15.46:0/1463896998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51a0102c80 0x7f51a0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 >> 172.21.15.46:0/1463896998 conn(0x7f51a0077990 msgr2=0x7f51a00fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 shutdown_connections 2024-09-12T02:52:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:24.459+0000 7f51a4e8b640 1 -- 172.21.15.46:0/1463896998 wait complete. 2024-09-12T02:52:24.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:24 smithi196 ceph-mon[34587]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:24 smithi046 ceph-mon[30951]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:24 smithi145 ceph-mon[34674]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:52:26.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:26 smithi196 ceph-mon[34587]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:26 smithi046 ceph-mon[30951]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:26 smithi145 ceph-mon[34674]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:28.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:28 smithi196 ceph-mon[34587]: pgmap v71: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:28 smithi046 ceph-mon[30951]: pgmap v71: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:28 smithi145 ceph-mon[34674]: pgmap v71: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:30.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:30 smithi196 ceph-mon[34587]: pgmap v72: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:30 smithi046 ceph-mon[30951]: pgmap v72: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:30 smithi145 ceph-mon[34674]: pgmap v72: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:32.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:32 smithi196 ceph-mon[34587]: pgmap v73: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:32.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:32 smithi046 ceph-mon[30951]: pgmap v73: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:32 smithi145 ceph-mon[34674]: pgmap v73: 129 pgs: 129 active+clean; 1.6 GiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:34.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:34 smithi196 ceph-mon[34587]: pgmap v74: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:34.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:34 smithi046 ceph-mon[30951]: pgmap v74: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:34 smithi145 ceph-mon[34674]: pgmap v74: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:36.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:36 smithi196 ceph-mon[34587]: pgmap v75: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:36 smithi046 ceph-mon[30951]: pgmap v75: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:36 smithi145 ceph-mon[34674]: pgmap v75: 129 pgs: 129 active+clean; 1.7 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 -- 172.21.15.46:0/4106296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241069c0 msgr2=0x7f3424106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 --2- 172.21.15.46:0/4106296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241069c0 0x7f3424106da0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3414009930 tx=0x7f341402a420 comp rx=0 tx=0).stop 2024-09-12T02:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 -- 172.21.15.46:0/4106296982 shutdown_connections 2024-09-12T02:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 --2- 172.21.15.46:0/4106296982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 0x7f342410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 --2- 172.21.15.46:0/4106296982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241009b0 0x7f3424100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 --2- 172.21.15.46:0/4106296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241069c0 0x7f3424106da0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 -- 172.21.15.46:0/4106296982 >> 172.21.15.46:0/4106296982 conn(0x7f3424077990 msgr2=0x7f34240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 -- 172.21.15.46:0/4106296982 shutdown_connections 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.559+0000 7f342a333640 1 -- 172.21.15.46:0/4106296982 wait complete. 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 Processor -- start 2024-09-12T02:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 -- start start 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 0x7f34241987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241069c0 0x7f342419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3424110cb0 con 0x7f34241009b0 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3424110b30 con 0x7f34241069c0 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f342a333640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3424110e30 con 0x7f3424101350 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f3423fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f3423fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59760/0 (socket says 172.21.15.46:59760) 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f3423fff640 1 -- 172.21.15.46:0/2649196709 learned_addr learned my addr 172.21.15.46:0/2649196709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f34288d0640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241069c0 0x7f342419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.560+0000 7f34237fe640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 0x7f34241987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 -- 172.21.15.46:0/2649196709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 msgr2=0x7f34241987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 0x7f34241987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 -- 172.21.15.46:0/2649196709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241069c0 msgr2=0x7f342419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241069c0 0x7f342419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 -- 172.21.15.46:0/2649196709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342419d2a0 con 0x7f34241009b0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f3423fff640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3414004770 tx=0x7f34140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f34217fa640 1 -- 172.21.15.46:0/2649196709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341403f070 con 0x7f34241009b0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f34217fa640 1 -- 172.21.15.46:0/2649196709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341403b460 con 0x7f34241009b0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342419d530 con 0x7f34241009b0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f34217fa640 1 -- 172.21.15.46:0/2649196709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414035070 con 0x7f34241009b0 2024-09-12T02:52:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.561+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a4e20 con 0x7f34241009b0 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f34217fa640 1 -- 172.21.15.46:0/2649196709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f341403b610 con 0x7f34241009b0 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 Processor -- start 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 -- start start 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f80063410 0x561f8006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f7ffc72f0 con 0x561f80062af0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561f7ff6f270 con 0x561f8006dcb0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f342d0c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f7ffc32b0 con 0x561f80063410 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f34217fa640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 0x7f340c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.564+0000 7f34288d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37800/0 (socket says 172.21.15.46:37800) 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 -- 172.21.15.46:0/172284946 learned_addr learned my addr 172.21.15.46:0/172284946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34217fa640 1 -- 172.21.15.46:0/2649196709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f34140efba0 con 0x7f34241009b0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34288d0640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 0x7f340c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f3423fff640 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f80063410 0x561f8006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 -- 172.21.15.46:0/172284946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f80063410 msgr2=0x561f8006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f80063410 0x561f8006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 -- 172.21.15.46:0/172284946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 msgr2=0x561f80062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80062ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 -- 172.21.15.46:0/172284946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7ff6e340 con 0x561f8006dcb0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34237fe640 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f341800bc80 tx=0x7f341800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac9f52cd6d7a59c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f34288d0640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 0x7f340c0a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f341c009fd0 tx=0x7f341c004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f340a7fc640 1 -- 172.21.15.46:0/172284946 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418015070 con 0x561f8006dcb0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f340a7fc640 1 -- 172.21.15.46:0/172284946 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341801a910 con 0x561f8006dcb0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.565+0000 7f340a7fc640 1 -- 172.21.15.46:0/172284946 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418011860 con 0x561f8006dcb0 2024-09-12T02:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 -- 172.21.15.46:0/172284946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 msgr2=0x561f800700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f341800bc80 tx=0x7f341800f730 comp rx=0 tx=0).stop 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 -- 172.21.15.46:0/172284946 shutdown_connections 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f8006dcb0 0x561f800700a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f80063410 0x561f8006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 --2- 172.21.15.46:0/172284946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80062ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 -- 172.21.15.46:0/172284946 >> 172.21.15.46:0/172284946 conn(0x561f8005f060 msgr2=0x561f8005f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 -- 172.21.15.46:0/172284946 shutdown_connections 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.566+0000 7f342d0c9740 1 -- 172.21.15.46:0/172284946 wait complete. 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 -- start start 2024-09-12T02:52:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 Processor -- start 2024-09-12T02:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f8006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f80070880 con 0x561f80063410 2024-09-12T02:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561f7ff50290 con 0x561f80062af0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f80070a00 con 0x561f8006dcb0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f342d0c9740 10 client.0 _ll_register_callbacks cb 0x561f7ff81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f34237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f34237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59776/0 (socket says 172.21.15.46:59776) 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f34237fe640 1 -- 172.21.15.46:0/2906956901 learned_addr learned my addr 172.21.15.46:0/2906956901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f3423fff640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.567+0000 7f34288d0640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f8006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 -- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 msgr2=0x561f8006b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f8006b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 -- 172.21.15.46:0/2906956901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 msgr2=0x561f8006abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 -- 172.21.15.46:0/2906956901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7fcda210 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34237fe640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f341800bdc0 tx=0x7f341800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1756620c732516b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34037fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418015070 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341801a910 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f341801a670 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f341802d050 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f342d0c9740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.568+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f7fda6620 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341801a910 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34180046c0 con 0x561f80063410 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 20 client.15525 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f8007dc40 0x561f80080030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 4 client.15525 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7ffd4a20 con 0x561f8007dc40 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.569+0000 7f3423fff640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f8007dc40 0x561f80080030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.570+0000 7f34037fe640 10 client.15525 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.570+0000 7f3423fff640 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f8007dc40 0x561f80080030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe4fee0069f012d server_cookie=d773ba80ffe759c7 in_seq=0 out_seq=0 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.570+0000 7f34037fe640 1 -- 172.21.15.46:0/2906956901 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f7ffd4a20 con 0x561f8007dc40 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.570+0000 7f34037fe640 10 client.15525 handle_command_reply: tid=0 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.570+0000 7f342d0c9740 1 client.15525 shutdown 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 msgr2=0x561f8006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f341800bdc0 tx=0x7f341800bdf0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 shutdown_connections 2024-09-12T02:52:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f8007dc40 0x561f80080030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f8006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006b110 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2906956901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 >> 172.21.15.46:0/2906956901 conn(0x561f8005f060 msgr2=0x561f80064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f34037fe640 0 client.15525 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 shutdown_connections 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 1 -- 172.21.15.46:0/2906956901 wait complete. 2024-09-12T02:52:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.571+0000 7f342d0c9740 20 client.15525 trim_cache size 0 max 16384 2024-09-12T02:52:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.589+0000 7f342d0c9740 1 Processor -- start 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 -- start start 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f80085630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b16c0 con 0x561f80063410 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34241aef60 con 0x561f80062af0 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f342d0c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34241acdb0 con 0x561f8006dcb0 2024-09-12T02:52:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f34237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f34288d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f80085630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f3423fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f34237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59790/0 (socket says 172.21.15.46:59790) 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f34288d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f80085630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:53698/0 (socket says 172.21.15.46:53698) 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.590+0000 7f34237fe640 1 -- 172.21.15.46:0/1069324564 learned_addr learned my addr 172.21.15.46:0/1069324564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 -- 172.21.15.46:0/1069324564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 msgr2=0x561f80085630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f80085630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 -- 172.21.15.46:0/1069324564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 msgr2=0x561f8006b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 -- 172.21.15.46:0/1069324564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f8005a2d0 con 0x561f80063410 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f3423fff640 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:52:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f34237fe640 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3418005d80 tx=0x7f3418005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8865d18687a8bbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f3409ffb640 1 -- 172.21.15.46:0/1069324564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418015040 con 0x561f80063410 2024-09-12T02:52:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f3409ffb640 1 -- 172.21.15.46:0/1069324564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341800be60 con 0x561f80063410 2024-09-12T02:52:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.591+0000 7f3409ffb640 1 -- 172.21.15.46:0/1069324564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180047b0 con 0x561f80063410 2024-09-12T02:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- 172.21.15.46:0/1069324564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 msgr2=0x561f8006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3418005d80 tx=0x7f3418005fb0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- 172.21.15.46:0/1069324564 shutdown_connections 2024-09-12T02:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f80085630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80063410 0x561f8006cb40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 --2- 172.21.15.46:0/1069324564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80062af0 0x561f8006b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- 172.21.15.46:0/1069324564 >> 172.21.15.46:0/1069324564 conn(0x561f8005f060 msgr2=0x561f8006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- 172.21.15.46:0/1069324564 shutdown_connections 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- 172.21.15.46:0/1069324564 wait complete. 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 -- start start 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.592+0000 7f342d0c9740 1 Processor -- start 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 0x561f80071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f800716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ad420 con 0x561f80062af0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ae890 con 0x561f80063410 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34241adaf0 con 0x561f8006dcb0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f342d0c9740 10 client.0 _ll_register_callbacks cb 0x7f341c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59794/0 (socket says 172.21.15.46:59794) 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 -- 172.21.15.46:0/2579284169 learned_addr learned my addr 172.21.15.46:0/2579284169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34288d0640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f800716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34237fe640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 0x561f80071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 -- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 msgr2=0x561f800716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f800716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 -- 172.21.15.46:0/2579284169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 msgr2=0x561f80071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 0x561f80071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 -- 172.21.15.46:0/2579284169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f800e6640 con 0x561f80062af0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34288d0640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f800716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34237fe640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 0x561f80071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f3423fff640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3414005940 tx=0x7f3414049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d5dd8ba8316d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34037fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.593+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140d9060 con 0x561f80062af0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f341403b940 con 0x561f80062af0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341800be60 con 0x561f80062af0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3414048040 con 0x561f80062af0 2024-09-12T02:52:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f800e7990 con 0x561f80062af0 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341800be60 con 0x561f80062af0 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f341404b060 con 0x561f80062af0 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 20 client.15531 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f800e80a0 0x561f800ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 4 client.15531 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f7ffced70 con 0x561f800e80a0 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.594+0000 7f34237fe640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f800e80a0 0x561f800ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.595+0000 7f34037fe640 10 client.15531 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.595+0000 7f34237fe640 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f800e80a0 0x561f800ea4c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cd6ce0401a1d87 server_cookie=a189ac4ea4faad14 in_seq=0 out_seq=0 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f34037fe640 1 -- 172.21.15.46:0/2579284169 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f7ffced70 con 0x561f800e80a0 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f34037fe640 10 client.15531 handle_command_reply: tid=0 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 client.15531 shutdown 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 msgr2=0x561f80070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3414005940 tx=0x7f3414049000 comp rx=0 tx=0).stop 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 shutdown_connections 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x561f800e80a0 0x561f800ea4c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f8006dcb0 0x561f800716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561f80063410 0x561f80071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f34037fe640 0 client.15531 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 --2- 172.21.15.46:0/2579284169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f80062af0 0x561f80070c50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 >> 172.21.15.46:0/2579284169 conn(0x561f8005f060 msgr2=0x561f8006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 shutdown_connections 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 1 -- 172.21.15.46:0/2579284169 wait complete. 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.603+0000 7f342d0c9740 20 client.15531 trim_cache size 0 max 16384 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:52:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:52:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:52:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 msgr2=0x7f340c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 0x7f340c0a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f341c009fd0 tx=0x7f341c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 msgr2=0x7f34241982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3414004770 tx=0x7f34140042f0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 shutdown_connections 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f340c0a2da0 0x7f340c0a5260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34241069c0 0x7f342419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3424101350 0x7f34241987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 --2- 172.21.15.46:0/2649196709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34241009b0 0x7f34241982b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 >> 172.21.15.46:0/2649196709 conn(0x7f3424077990 msgr2=0x7f3424069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 shutdown_connections 2024-09-12T02:52:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:37.604+0000 7f342a333640 1 -- 172.21.15.46:0/2649196709 wait complete. 2024-09-12T02:52:38.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:38 smithi196 ceph-mon[34587]: pgmap v76: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:38.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:38 smithi046 ceph-mon[30951]: pgmap v76: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:38 smithi145 ceph-mon[34674]: pgmap v76: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:40.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:40 smithi196 ceph-mon[34587]: pgmap v77: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:40.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:40 smithi046 ceph-mon[30951]: pgmap v77: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:40.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:40 smithi145 ceph-mon[34674]: pgmap v77: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:42.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:42 smithi196 ceph-mon[34587]: pgmap v78: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:42.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:42 smithi046 ceph-mon[30951]: pgmap v78: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:42 smithi145 ceph-mon[34674]: pgmap v78: 129 pgs: 129 active+clean; 1.8 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:52:44.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:44 smithi196 ceph-mon[34587]: pgmap v79: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:44.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:44 smithi046 ceph-mon[30951]: pgmap v79: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:44 smithi145 ceph-mon[34674]: pgmap v79: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:46.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:46 smithi196 ceph-mon[34587]: pgmap v80: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:46.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:46 smithi046 ceph-mon[30951]: pgmap v80: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:46 smithi145 ceph-mon[34674]: pgmap v80: 129 pgs: 129 active+clean; 1.9 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:48 smithi196 ceph-mon[34587]: pgmap v81: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-12T02:52:48.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:48 smithi046 ceph-mon[30951]: pgmap v81: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-12T02:52:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:48 smithi145 ceph-mon[34674]: pgmap v81: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 23 op/s 2024-09-12T02:52:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:50 smithi196 ceph-mon[34587]: pgmap v82: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:50.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:50 smithi046 ceph-mon[30951]: pgmap v82: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:50 smithi145 ceph-mon[34674]: pgmap v82: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 -- 172.21.15.46:0/3333386827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc101350 msgr2=0x7fabdc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/3333386827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc101350 0x7fabdc10d880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009cd0 tx=0x7fabd402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 -- 172.21.15.46:0/3333386827 shutdown_connections 2024-09-12T02:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/3333386827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc101350 0x7fabdc10d880 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/3333386827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc1009b0 0x7fabdc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/3333386827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 -- 172.21.15.46:0/3333386827 >> 172.21.15.46:0/3333386827 conn(0x7fabdc077990 msgr2=0x7fabdc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 -- 172.21.15.46:0/3333386827 shutdown_connections 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.703+0000 7fabe0f4b640 1 -- 172.21.15.46:0/3333386827 wait complete. 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 Processor -- start 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 -- start start 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 0x7fabdc197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc110cb0 con 0x7fabdc1009b0 2024-09-12T02:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc110b30 con 0x7fabdc101350 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabe0f4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc110e30 con 0x7fabdc1069c0 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49150/0 (socket says 172.21.15.46:49150) 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 -- 172.21.15.46:0/2266449716 learned_addr learned my addr 172.21.15.46:0/2266449716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabdad9d640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 0x7fabdc197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 -- 172.21.15.46:0/2266449716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 msgr2=0x7fabdc19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 -- 172.21.15.46:0/2266449716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 msgr2=0x7fabdc197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.704+0000 7fabda59c640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 0x7fabdc197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabda59c640 1 -- 172.21.15.46:0/2266449716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc19c140 con 0x7fabdc1009b0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 0x7fabdc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabdad9d640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabda59c640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009860 tx=0x7fabcc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabcb7fe640 1 -- 172.21.15.46:0/2266449716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc005020 con 0x7fabdc1009b0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabcb7fe640 1 -- 172.21.15.46:0/2266449716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabcc0047a0 con 0x7fabdc1009b0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc19c430 con 0x7fabdc1009b0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabcb7fe640 1 -- 172.21.15.46:0/2266449716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc007b00 con 0x7fabdc1009b0 2024-09-12T02:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.705+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc0698a0 con 0x7fabdc1009b0 2024-09-12T02:52:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.707+0000 7fabe3dc3740 1 Processor -- start 2024-09-12T02:52:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 -- start start 2024-09-12T02:52:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabcb7fe640 1 -- 172.21.15.46:0/2266449716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fabcc007ca0 con 0x7fabdc1009b0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c9ede912f0 con 0x55c9edf2c9f0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c9ede39270 con 0x55c9edf2d310 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9ede8d2b0 con 0x55c9edf3c0c0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49152/0 (socket says 172.21.15.46:49152) 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabdad9d640 1 -- 172.21.15.46:0/1799059669 learned_addr learned my addr 172.21.15.46:0/1799059669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabda59c640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.708+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 -- 172.21.15.46:0/1799059669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 msgr2=0x55c9edf3e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf3e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 -- 172.21.15.46:0/1799059669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 msgr2=0x55c9edf3bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf3bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 -- 172.21.15.46:0/1799059669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ede38340 con 0x55c9edf2c9f0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005cc0 tx=0x7fabd4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912250de8352b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabcb7fe640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 0x7fabc40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabc8ff9640 1 -- 172.21.15.46:0/1799059669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4031a50 con 0x55c9edf2c9f0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabc8ff9640 1 -- 172.21.15.46:0/1799059669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabc8ff9640 1 -- 172.21.15.46:0/1799059669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004600 con 0x55c9edf2c9f0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabda59c640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 0x7fabc40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.709+0000 7fabcb7fe640 1 -- 172.21.15.46:0/2266449716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fabcc0ceae0 con 0x7fabdc1009b0 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1799059669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 msgr2=0x55c9edf2cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabda59c640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 0x7fabc40a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0cbf80 tx=0x7fabcc0d2620 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005cc0 tx=0x7fabd4009a20 comp rx=0 tx=0).stop 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1799059669 shutdown_connections 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf3bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1799059669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2cdd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1799059669 >> 172.21.15.46:0/1799059669 conn(0x55c9edf29060 msgr2=0x55c9edf29440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1799059669 shutdown_connections 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1799059669 wait complete. 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.710+0000 7fabe3dc3740 1 -- start start 2024-09-12T02:52:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 Processor -- start 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c9edf3ecc0 con 0x55c9edf2c9f0 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c9ede1a290 con 0x55c9edf3c0c0 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9edf3ee40 con 0x55c9edf2d310 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabe3dc3740 10 client.0 _ll_register_callbacks cb 0x55c9ede4b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49156/0 (socket says 172.21.15.46:49156) 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 -- 172.21.15.46:0/1140692392 learned_addr learned my addr 172.21.15.46:0/1140692392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 -- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 msgr2=0x55c9edf39570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf39570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 -- 172.21.15.46:0/1140692392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 msgr2=0x55c9edf39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf39ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabdad9d640 1 -- 172.21.15.46:0/1140692392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9edba4210 con 0x55c9edf2c9f0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.711+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabdad9d640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009620 tx=0x7fabd402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0c990eb1875 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4003970 con 0x55c9edf2c9f0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fabd4003b10 con 0x55c9edf2c9f0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabd4003d10 con 0x55c9edf2c9f0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabe3dc3740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9edc70620 con 0x55c9edf2c9f0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabd4030070 con 0x55c9edf2c9f0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.712+0000 7fabe3dc3740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabe3dc3740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabe3dc3740 20 client.15543 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edf4bf40 0x55c9edf4e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabe3dc3740 4 client.15543 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ede9ea20 con 0x55c9edf4bf40 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edf4bf40 0x55c9edf4e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edf4bf40 0x55c9edf4e330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7251d72cf01166 server_cookie=46830e21f21ecac1 in_seq=0 out_seq=0 2024-09-12T02:52:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.713+0000 7fabc1ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabc1ffb640 1 -- 172.21.15.46:0/1140692392 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9ede9ea20 con 0x55c9edf4bf40 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabc1ffb640 10 client.15543 handle_command_reply: tid=0 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 client.15543 shutdown 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 msgr2=0x55c9edf39030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009620 tx=0x7fabd402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 shutdown_connections 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edf4bf40 0x55c9edf4e330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabc1ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/1140692392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf39030 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 >> 172.21.15.46:0/1140692392 conn(0x55c9edf29060 msgr2=0x55c9edf2edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 shutdown_connections 2024-09-12T02:52:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 1 -- 172.21.15.46:0/1140692392 wait complete. 2024-09-12T02:52:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.714+0000 7fabe3dc3740 20 client.15543 trim_cache size 0 max 16384 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 Processor -- start 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 -- start start 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf53e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1af520 con 0x55c9edf2c9f0 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1acdc0 con 0x55c9edf2d310 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.729+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1aac10 con 0x55c9edf3c0c0 2024-09-12T02:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49164/0 (socket says 172.21.15.46:49164) 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 -- 172.21.15.46:0/4018741848 learned_addr learned my addr 172.21.15.46:0/4018741848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 -- 172.21.15.46:0/4018741848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 msgr2=0x55c9edf53e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf53e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 -- 172.21.15.46:0/4018741848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 msgr2=0x55c9edf538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 -- 172.21.15.46:0/4018741848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9edf2d140 con 0x55c9edf2c9f0 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabdad9d640 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabd4000f80 tx=0x7fabd4046640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b871dcc114e35d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabc3fff640 1 -- 172.21.15.46:0/4018741848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd404a040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabc3fff640 1 -- 172.21.15.46:0/4018741848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.730+0000 7fabc3fff640 1 -- 172.21.15.46:0/4018741848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd40314b0 con 0x55c9edf2c9f0 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- 172.21.15.46:0/4018741848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 msgr2=0x55c9edf2ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabd4000f80 tx=0x7fabd4046640 comp rx=0 tx=0).stop 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- 172.21.15.46:0/4018741848 shutdown_connections 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf3c0c0 0x55c9edf53e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf2d310 0x55c9edf538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/4018741848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf2ecb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- 172.21.15.46:0/4018741848 >> 172.21.15.46:0/4018741848 conn(0x55c9edf29060 msgr2=0x55c9edf2e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- 172.21.15.46:0/4018741848 shutdown_connections 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- 172.21.15.46:0/4018741848 wait complete. 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 -- start start 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.731+0000 7fabe3dc3740 1 Processor -- start 2024-09-12T02:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1ab280 con 0x55c9edf2c9f0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1ac6f0 con 0x55c9edf3c0c0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1ab950 con 0x55c9edf2d310 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabe3dc3740 10 client.0 _ll_register_callbacks cb 0x7fabcc0d24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49170/0 (socket says 172.21.15.46:49170) 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 -- 172.21.15.46:0/3360907911 learned_addr learned my addr 172.21.15.46:0/3360907911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 -- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 msgr2=0x55c9edf403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 -- 172.21.15.46:0/3360907911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 msgr2=0x55c9edf408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 -- 172.21.15.46:0/3360907911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9edfb58a0 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabdad9d640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fabd402aff0 tx=0x7fabd4065040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff504cabe47b686 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.732+0000 7fabc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd404a040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fabd402e070 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabd4073050 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9edfb6b00 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd4036040 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabd406e030 con 0x55c9edf2c9f0 2024-09-12T02:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 20 client.15549 populate_metadata read hostname 'smithi046' 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edfb70a0 0x55c9edfb94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 4 client.15549 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9ede98d70 con 0x55c9edfb70a0 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.733+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edfb70a0 0x55c9edfb94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.734+0000 7fabc1ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.734+0000 7fabd9d9b640 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edfb70a0 0x55c9edfb94c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b615bff1c3968a1 server_cookie=e39aa5ae6d7fa9e5 in_seq=0 out_seq=0 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.741+0000 7fabc1ffb640 1 -- 172.21.15.46:0/3360907911 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9ede98d70 con 0x55c9edfb70a0 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.741+0000 7fabc1ffb640 10 client.15549 handle_command_reply: tid=0 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 client.15549 shutdown 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 msgr2=0x55c9edf3fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fabd402aff0 tx=0x7fabd4065040 comp rx=0 tx=0).stop 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 shutdown_connections 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c9edfb70a0 0x55c9edfb94c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c9edf3c0c0 0x55c9edf408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9edf2d310 0x55c9edf403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabc1ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 --2- 172.21.15.46:0/3360907911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9edf2c9f0 0x55c9edf3fe60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 >> 172.21.15.46:0/3360907911 conn(0x55c9edf29060 msgr2=0x55c9edf2edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 shutdown_connections 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 1 -- 172.21.15.46:0/3360907911 wait complete. 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe3dc3740 20 client.15549 trim_cache size 0 max 16384 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:52:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 msgr2=0x7fabc40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 0x7fabc40a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0cbf80 tx=0x7fabcc0d2620 comp rx=0 tx=0).stop 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 msgr2=0x7fabdc197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.742+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009860 tx=0x7fabcc009d20 comp rx=0 tx=0).stop 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 shutdown_connections 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fabc40a2da0 0x7fabc40a5260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabdc1069c0 0x7fabdc19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabdc101350 0x7fabdc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 --2- 172.21.15.46:0/2266449716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabdc1009b0 0x7fabdc197150 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 >> 172.21.15.46:0/2266449716 conn(0x7fabdc077990 msgr2=0x7fabdc1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 shutdown_connections 2024-09-12T02:52:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:52:50.743+0000 7fabe0f4b640 1 -- 172.21.15.46:0/2266449716 wait complete. 2024-09-12T02:52:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:52 smithi196 ceph-mon[34587]: pgmap v83: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:52.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:52 smithi046 ceph-mon[30951]: pgmap v83: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:52 smithi145 ceph-mon[34674]: pgmap v83: 129 pgs: 129 active+clean; 2.1 GiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:54 smithi046 ceph-mon[30951]: pgmap v84: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:52:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:54 smithi145 ceph-mon[34674]: pgmap v84: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:52:54.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:54 smithi196 ceph-mon[34587]: pgmap v84: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:52:56.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:56 smithi046 ceph-mon[30951]: pgmap v85: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:56 smithi145 ceph-mon[34674]: pgmap v85: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:56 smithi196 ceph-mon[34587]: pgmap v85: 129 pgs: 129 active+clean; 2.2 GiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:52:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:52:58 smithi046 ceph-mon[30951]: pgmap v86: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:52:58 smithi145 ceph-mon[34674]: pgmap v86: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:52:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:52:58 smithi196 ceph-mon[34587]: pgmap v86: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:00 smithi046 ceph-mon[30951]: pgmap v87: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:00 smithi145 ceph-mon[34674]: pgmap v87: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:00 smithi196 ceph-mon[34587]: pgmap v87: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:02 smithi046 ceph-mon[30951]: pgmap v88: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:02 smithi145 ceph-mon[34674]: pgmap v88: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:02 smithi196 ceph-mon[34587]: pgmap v88: 129 pgs: 129 active+clean; 2.3 GiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 -- 172.21.15.46:0/3922913738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c107b70 msgr2=0x7f652c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 --2- 172.21.15.46:0/3922913738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c107b70 0x7f652c107f50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6528009930 tx=0x7f652802a420 comp rx=0 tx=0).stop 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 -- 172.21.15.46:0/3922913738 shutdown_connections 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 --2- 172.21.15.46:0/3922913738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c102470 0x7f652c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 --2- 172.21.15.46:0/3922913738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 --2- 172.21.15.46:0/3922913738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c107b70 0x7f652c107f50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.844+0000 7f653405e640 1 -- 172.21.15.46:0/3922913738 >> 172.21.15.46:0/3922913738 conn(0x7f652c0fd7d0 msgr2=0x7f652c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.845+0000 7f653405e640 1 -- 172.21.15.46:0/3922913738 shutdown_connections 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.845+0000 7f653405e640 1 -- 172.21.15.46:0/3922913738 wait complete. 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.845+0000 7f653405e640 1 Processor -- start 2024-09-12T02:53:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.845+0000 7f653405e640 1 -- start start 2024-09-12T02:53:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.845+0000 7f653405e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 0x7f652c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f652c111de0 con 0x7f652c102470 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f652c111c60 con 0x7f652c101ad0 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c111f60 con 0x7f652c107b70 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34232/0 (socket says 172.21.15.46:34232) 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 -- 172.21.15.46:0/4032004276 learned_addr learned my addr 172.21.15.46:0/4032004276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65325fb640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 0x7f652c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f6531dfa640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 -- 172.21.15.46:0/4032004276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 msgr2=0x7f652c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 0x7f652c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 -- 172.21.15.46:0/4032004276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 msgr2=0x7f652c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 -- 172.21.15.46:0/4032004276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c19d290 con 0x7f652c102470 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65325fb640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 0x7f652c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f6531dfa640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f65315f9640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f651c00e9f0 tx=0x7f651c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f6522ffd640 1 -- 172.21.15.46:0/4032004276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c00cd80 con 0x7f652c102470 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f6522ffd640 1 -- 172.21.15.46:0/4032004276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f651c0108e0 con 0x7f652c102470 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c19d580 con 0x7f652c102470 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.846+0000 7f6522ffd640 1 -- 172.21.15.46:0/4032004276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c018920 con 0x7f652c102470 2024-09-12T02:53:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.847+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c1a4e10 con 0x7f652c102470 2024-09-12T02:53:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.849+0000 7f6536df7740 1 Processor -- start 2024-09-12T02:53:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 -- start start 2024-09-12T02:53:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6522ffd640 1 -- 172.21.15.46:0/4032004276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f651c010a90 con 0x7f652c102470 2024-09-12T02:53:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a8584f0c0 0x559a858514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a857a42f0 con 0x559a85843f00 2024-09-12T02:53:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a8574c270 con 0x559a8584f0c0 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6536df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a857a02b0 con 0x559a85844820 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34242/0 (socket says 172.21.15.46:34242) 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 -- 172.21.15.46:0/679070220 learned_addr learned my addr 172.21.15.46:0/679070220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f6531dfa640 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65315f9640 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a8584f0c0 0x559a858514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 -- 172.21.15.46:0/679070220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 msgr2=0x559a8584eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 -- 172.21.15.46:0/679070220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a8584f0c0 msgr2=0x559a858514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a8584f0c0 0x559a858514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.850+0000 7f65325fb640 1 -- 172.21.15.46:0/679070220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8574b340 con 0x559a85843f00 2024-09-12T02:53:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f65325fb640 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6524009f10 tx=0x7f652400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988f93ca9223c011 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6522ffd640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 0x7f65180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6517fff640 1 -- 172.21.15.46:0/679070220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524003b70 con 0x559a85843f00 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6531dfa640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 0x7f65180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6517fff640 1 -- 172.21.15.46:0/679070220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6524003d10 con 0x559a85843f00 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6517fff640 1 -- 172.21.15.46:0/679070220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524011a00 con 0x559a85843f00 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.851+0000 7f6522ffd640 1 -- 172.21.15.46:0/4032004276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f651c0cfbe0 con 0x7f652c102470 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- 172.21.15.46:0/679070220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 msgr2=0x559a858442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6524009f10 tx=0x7f652400f730 comp rx=0 tx=0).stop 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6531dfa640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 0x7f65180a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6528009230 tx=0x7f6528005b50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- 172.21.15.46:0/679070220 shutdown_connections 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a8584f0c0 0x559a858514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 --2- 172.21.15.46:0/679070220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a858442e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- 172.21.15.46:0/679070220 >> 172.21.15.46:0/679070220 conn(0x559a85840470 msgr2=0x559a85840850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- 172.21.15.46:0/679070220 shutdown_connections 2024-09-12T02:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- 172.21.15.46:0/679070220 wait complete. 2024-09-12T02:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 -- start start 2024-09-12T02:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.852+0000 7f6536df7740 1 Processor -- start 2024-09-12T02:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a8584bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a85851c90 con 0x559a8584f0c0 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559a8572d290 con 0x559a85843f00 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a85851e10 con 0x559a85844820 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6536df7740 10 client.0 _ll_register_callbacks cb 0x559a8575e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34252/0 (socket says 172.21.15.46:34252) 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 -- 172.21.15.46:0/1761148925 learned_addr learned my addr 172.21.15.46:0/1761148925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65315f9640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f6531dfa640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a8584bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 -- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 msgr2=0x559a8584c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 -- 172.21.15.46:0/1761148925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 msgr2=0x559a8584bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a8584bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.853+0000 7f65325fb640 1 -- 172.21.15.46:0/1761148925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a854b7210 con 0x559a8584f0c0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65325fb640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6524004900 tx=0x7f6524009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44478bc6596c636 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524004a40 con 0x559a8584f0c0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6524003d10 con 0x559a8584f0c0 2024-09-12T02:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6524004be0 con 0x559a8584f0c0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6524030050 con 0x559a8584f0c0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a85583620 con 0x559a8584f0c0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6524003d10 con 0x559a8584f0c0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f65240041b0 con 0x559a8584f0c0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 20 client.15561 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a8585efd0 0x559a858613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 4 client.15561 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.854+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a857b1a20 con 0x559a8585efd0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.855+0000 7f6531dfa640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a8585efd0 0x559a858613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.855+0000 7f6531dfa640 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a8585efd0 0x559a858613c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a85b9f8d8d2ec4e server_cookie=c4b625db92099e37 in_seq=0 out_seq=0 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.855+0000 7f65157fa640 10 client.15561 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f65157fa640 1 -- 172.21.15.46:0/1761148925 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a857b1a20 con 0x559a8585efd0 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f65157fa640 10 client.15561 handle_command_reply: tid=0 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 client.15561 shutdown 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 msgr2=0x559a8584ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6524004900 tx=0x7f6524009cd0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 shutdown_connections 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a8585efd0 0x559a858613c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a8584f0c0 0x559a8584ca40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a85844820 0x559a8584c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 --2- 172.21.15.46:0/1761148925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a8584bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f65157fa640 0 client.15561 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 >> 172.21.15.46:0/1761148925 conn(0x559a85840470 msgr2=0x559a8583b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 shutdown_connections 2024-09-12T02:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 1 -- 172.21.15.46:0/1761148925 wait complete. 2024-09-12T02:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.856+0000 7f6536df7740 20 client.15561 trim_cache size 0 max 16384 2024-09-12T02:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 Processor -- start 2024-09-12T02:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 -- start start 2024-09-12T02:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 0x559a8584d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85866990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1af570 con 0x559a85843f00 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ace10 con 0x559a85844820 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.879+0000 7f6536df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1aac60 con 0x559a8584f0c0 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34268/0 (socket says 172.21.15.46:34268) 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 -- 172.21.15.46:0/3524547749 learned_addr learned my addr 172.21.15.46:0/3524547749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65315f9640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 0x559a8584d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65325fb640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85866990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 -- 172.21.15.46:0/3524547749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 msgr2=0x559a85866990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85866990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 -- 172.21.15.46:0/3524547749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 msgr2=0x559a8584d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 0x559a8584d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 -- 172.21.15.46:0/3524547749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a85844650 con 0x559a85843f00 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65315f9640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 0x559a8584d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f6531dfa640 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6528009930 tx=0x7f6528040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9849243b72e8344 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65177fe640 1 -- 172.21.15.46:0/3524547749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528035040 con 0x559a85843f00 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65177fe640 1 -- 172.21.15.46:0/3524547749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6528004d70 con 0x559a85843f00 2024-09-12T02:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.880+0000 7f65177fe640 1 -- 172.21.15.46:0/3524547749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528003e20 con 0x559a85843f00 2024-09-12T02:53:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 -- 172.21.15.46:0/3524547749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 msgr2=0x559a8584b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6528009930 tx=0x7f6528040000 comp rx=0 tx=0).stop 2024-09-12T02:53:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 -- 172.21.15.46:0/3524547749 shutdown_connections 2024-09-12T02:53:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85866990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85844820 0x559a8584d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 --2- 172.21.15.46:0/3524547749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85843f00 0x559a8584b880 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 -- 172.21.15.46:0/3524547749 >> 172.21.15.46:0/3524547749 conn(0x559a85840470 msgr2=0x559a8583b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 -- 172.21.15.46:0/3524547749 shutdown_connections 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.881+0000 7f6536df7740 1 -- 172.21.15.46:0/3524547749 wait complete. 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 -- start start 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 Processor -- start 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a858573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85852830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ab2d0 con 0x559a85844820 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ac740 con 0x559a85843f00 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ab9a0 con 0x559a8584f0c0 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f6536df7740 10 client.0 _ll_register_callbacks cb 0x7f6524015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f65315f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f65315f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34282/0 (socket says 172.21.15.46:34282) 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.882+0000 7f65315f9640 1 -- 172.21.15.46:0/2069558892 learned_addr learned my addr 172.21.15.46:0/2069558892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65325fb640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85852830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f6531dfa640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a858573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 -- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 msgr2=0x559a85852830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85852830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 -- 172.21.15.46:0/2069558892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 msgr2=0x559a858573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a858573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 -- 172.21.15.46:0/2069558892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a85853030 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65315f9640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c6a0 tx=0x7f651c0d32f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcce7b743cef43f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65157fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c09f020 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6528004d70 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f651c010430 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f651c09b700 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f6536df7740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a858c95e0 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.883+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6528004d70 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f651c097af0 con 0x559a85844820 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 20 client.15567 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a858c9d90 0x559a858cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 4 client.15567 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a857abd70 con 0x559a858c9d90 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6531dfa640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a858c9d90 0x559a858cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f65157fa640 10 client.15567 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.884+0000 7f6531dfa640 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a858c9d90 0x559a858cc1b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a339adc5b29e55b server_cookie=6458635cbc66bd69 in_seq=0 out_seq=0 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f65157fa640 1 -- 172.21.15.46:0/2069558892 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a857abd70 con 0x559a858c9d90 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f65157fa640 10 client.15567 handle_command_reply: tid=0 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 client.15567 shutdown 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 msgr2=0x559a858578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c6a0 tx=0x7f651c0d32f0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 shutdown_connections 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559a858c9d90 0x559a858cc1b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a8584f0c0 0x559a85852830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a85844820 0x559a858578e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 --2- 172.21.15.46:0/2069558892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559a85843f00 0x559a858573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f65157fa640 0 client.15567 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 >> 172.21.15.46:0/2069558892 conn(0x559a85840470 msgr2=0x559a8584c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 shutdown_connections 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 1 -- 172.21.15.46:0/2069558892 wait complete. 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.893+0000 7f6536df7740 20 client.15567 trim_cache size 0 max 16384 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:53:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:53:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 msgr2=0x7f65180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 0x7f65180a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6528009230 tx=0x7f6528005b50 comp rx=0 tx=0).stop 2024-09-12T02:53:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 msgr2=0x7f652c1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f651c00e9f0 tx=0x7f651c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 shutdown_connections 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f65180a2da0 0x7f65180a5260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c107b70 0x7f652c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f652c102470 0x7f652c1987e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 --2- 172.21.15.46:0/4032004276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f652c101ad0 0x7f652c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 >> 172.21.15.46:0/4032004276 conn(0x7f652c0fd7d0 msgr2=0x7f652c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 shutdown_connections 2024-09-12T02:53:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:03.894+0000 7f653405e640 1 -- 172.21.15.46:0/4032004276 wait complete. 2024-09-12T02:53:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:04 smithi046 ceph-mon[30951]: pgmap v89: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:04 smithi145 ceph-mon[34674]: pgmap v89: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:04 smithi196 ceph-mon[34587]: pgmap v89: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:06 smithi046 ceph-mon[30951]: pgmap v90: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:53:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:06 smithi145 ceph-mon[34674]: pgmap v90: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:53:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:06 smithi196 ceph-mon[34587]: pgmap v90: 129 pgs: 129 active+clean; 2.5 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:53:08.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:08 smithi046 ceph-mon[30951]: pgmap v91: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:53:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:08 smithi145 ceph-mon[34674]: pgmap v91: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:53:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:08 smithi196 ceph-mon[34587]: pgmap v91: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:53:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:10 smithi046 ceph-mon[30951]: pgmap v92: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:10 smithi145 ceph-mon[34674]: pgmap v92: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:10 smithi196 ceph-mon[34587]: pgmap v92: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:12 smithi046 ceph-mon[30951]: pgmap v93: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:12 smithi145 ceph-mon[34674]: pgmap v93: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:12 smithi196 ceph-mon[34587]: pgmap v93: 129 pgs: 129 active+clean; 2.6 GiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:53:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:14 smithi046 ceph-mon[30951]: pgmap v94: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:14 smithi145 ceph-mon[34674]: pgmap v94: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:14 smithi196 ceph-mon[34587]: pgmap v94: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:53:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:16 smithi046 ceph-mon[30951]: pgmap v95: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:16 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:16 smithi145 ceph-mon[34674]: pgmap v95: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:16 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:16 smithi196 ceph-mon[34587]: pgmap v95: 129 pgs: 129 active+clean; 2.7 GiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:16 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 -- 172.21.15.46:0/1954224575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34103600 msgr2=0x7f6f3410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 --2- 172.21.15.46:0/1954224575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34103600 0x7f6f3410bae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a480 comp rx=0 tx=0).stop 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 -- 172.21.15.46:0/1954224575 shutdown_connections 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 --2- 172.21.15.46:0/1954224575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3410c0b0 0x7f6f3410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 --2- 172.21.15.46:0/1954224575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34103600 0x7f6f3410bae0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 --2- 172.21.15.46:0/1954224575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f34102ce0 0x7f6f341030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 -- 172.21.15.46:0/1954224575 >> 172.21.15.46:0/1954224575 conn(0x7f6f34077990 msgr2=0x7f6f340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 -- 172.21.15.46:0/1954224575 shutdown_connections 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 -- 172.21.15.46:0/1954224575 wait complete. 2024-09-12T02:53:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.994+0000 7f6f39c47640 1 Processor -- start 2024-09-12T02:53:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 -- start start 2024-09-12T02:53:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f34103600 0x7f6f34198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 0x7f6f3419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111d60 con 0x7f6f34102ce0 2024-09-12T02:53:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111be0 con 0x7f6f3410c0b0 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f39c47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111ee0 con 0x7f6f34103600 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57456/0 (socket says 172.21.15.46:57456) 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 -- 172.21.15.46:0/692924970 learned_addr learned my addr 172.21.15.46:0/692924970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f34103600 0x7f6f34198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f33fff640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 0x7f6f3419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 -- 172.21.15.46:0/692924970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f34103600 msgr2=0x7f6f34198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f34103600 0x7f6f34198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 -- 172.21.15.46:0/692924970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 msgr2=0x7f6f3419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 0x7f6f3419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f337fe640 1 -- 172.21.15.46:0/692924970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419d210 con 0x7f6f34102ce0 2024-09-12T02:53:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.995+0000 7f6f33fff640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 0x7f6f3419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f337fe640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c920 tx=0x7f6f2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f30ff9640 1 -- 172.21.15.46:0/692924970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24007c70 con 0x7f6f34102ce0 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3419d500 con 0x7f6f34102ce0 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f30ff9640 1 -- 172.21.15.46:0/692924970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f24007e10 con 0x7f6f34102ce0 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f30ff9640 1 -- 172.21.15.46:0/692924970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24018820 con 0x7f6f34102ce0 2024-09-12T02:53:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.996+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a4d90 con 0x7f6f34102ce0 2024-09-12T02:53:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.998+0000 7f6f30ff9640 1 -- 172.21.15.46:0/692924970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f240189c0 con 0x7f6f34102ce0 2024-09-12T02:53:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.998+0000 7f6f3c9db740 1 Processor -- start 2024-09-12T02:53:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 -- start start 2024-09-12T02:53:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955954af0 0x559955954ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x5599559620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5599558b92f0 con 0x559955955410 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559955861270 con 0x55995595fcb0 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599558b52b0 con 0x559955954af0 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57458/0 (socket says 172.21.15.46:57458) 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 -- 172.21.15.46:0/2570561206 learned_addr learned my addr 172.21.15.46:0/2570561206 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f33fff640 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955954af0 0x559955954ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x5599559620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f30ff9640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 0x7f6f1c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 -- 172.21.15.46:0/2570561206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955954af0 msgr2=0x559955954ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955954af0 0x559955954ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 -- 172.21.15.46:0/2570561206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 msgr2=0x5599559620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x5599559620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:16.999+0000 7f6f337fe640 1 -- 172.21.15.46:0/2570561206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559955860340 con 0x559955955410 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f33fff640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 0x7f6f1c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f30ff9640 1 -- 172.21.15.46:0/692924970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6f240cf8e0 con 0x7f6f34102ce0 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f337fe640 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6f240ccc40 tx=0x7f6f24015d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca575f2032f23f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f33fff640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 0x7f6f1c0a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005fd0 tx=0x7f6f2c004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f19ffb640 1 -- 172.21.15.46:0/2570561206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2409b5e0 con 0x559955955410 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f19ffb640 1 -- 172.21.15.46:0/2570561206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f24066b30 con 0x559955955410 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.000+0000 7f6f19ffb640 1 -- 172.21.15.46:0/2570561206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24018cb0 con 0x559955955410 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- 172.21.15.46:0/2570561206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 msgr2=0x55995595f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6f240ccc40 tx=0x7f6f24015d00 comp rx=0 tx=0).stop 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- 172.21.15.46:0/2570561206 shutdown_connections 2024-09-12T02:53:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x5599559620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595f770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/2570561206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955954af0 0x559955954ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- 172.21.15.46:0/2570561206 >> 172.21.15.46:0/2570561206 conn(0x559955951060 msgr2=0x559955951440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- 172.21.15.46:0/2570561206 shutdown_connections 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- 172.21.15.46:0/2570561206 wait complete. 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.001+0000 7f6f3c9db740 1 -- start start 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 Processor -- start 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955954af0 0x55995595cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x55995595d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559955962880 con 0x559955955410 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559955842290 con 0x559955954af0 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559955962a00 con 0x55995595fcb0 2024-09-12T02:53:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f3c9db740 10 client.0 _ll_register_callbacks cb 0x559955873140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57472/0 (socket says 172.21.15.46:57472) 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 -- 172.21.15.46:0/1654577177 learned_addr learned my addr 172.21.15.46:0/1654577177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955954af0 0x55995595cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f33fff640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x55995595d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 -- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 msgr2=0x55995595d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x55995595d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 -- 172.21.15.46:0/1654577177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955954af0 msgr2=0x55995595cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955954af0 0x55995595cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.002+0000 7f6f32ffd640 1 -- 172.21.15.46:0/1654577177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599555cc210 con 0x559955955410 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6f28033000 tx=0x7f6f28031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d20208efbf44e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035040 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f24066b30 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6f28004050 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f28004250 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f3c9db740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559955698620 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.003+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f24066b30 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6f28004a70 con 0x559955955410 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 20 client.15579 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55995596fbc0 0x559955972040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 4 client.15579 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:53:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599558c6a20 con 0x55995596fbc0 2024-09-12T02:53:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55995596fbc0 0x559955972040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f12ffd640 10 client.15579 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.004+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55995596fbc0 0x559955972040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d98b35431e873f0 server_cookie=266b57ec89061a92 in_seq=0 out_seq=0 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f12ffd640 1 -- 172.21.15.46:0/1654577177 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599558c6a20 con 0x55995596fbc0 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f12ffd640 10 client.15579 handle_command_reply: tid=0 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 client.15579 shutdown 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 msgr2=0x55995595d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6f28033000 tx=0x7f6f28031f40 comp rx=0 tx=0).stop 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 shutdown_connections 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55995596fbc0 0x559955972040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x55995595d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955955410 0x55995595d0a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1654577177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955954af0 0x55995595cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 >> 172.21.15.46:0/1654577177 conn(0x559955951060 msgr2=0x559955956dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f12ffd640 0 client.15579 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 shutdown_connections 2024-09-12T02:53:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1654577177 wait complete. 2024-09-12T02:53:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.005+0000 7f6f3c9db740 20 client.15579 trim_cache size 0 max 16384 2024-09-12T02:53:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 Processor -- start 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 -- start start 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955955410 0x559955957e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x55995595d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b16e0 con 0x559955954af0 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341aef80 con 0x55995595fcb0 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341acdd0 con 0x559955955410 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57482/0 (socket says 172.21.15.46:57482) 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f337fe640 1 -- 172.21.15.46:0/1362355612 learned_addr learned my addr 172.21.15.46:0/1362355612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955955410 0x559955957e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.020+0000 7f6f33fff640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x55995595d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 -- 172.21.15.46:0/1362355612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955955410 msgr2=0x559955957e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955955410 0x559955957e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 -- 172.21.15.46:0/1362355612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 msgr2=0x55995595d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x55995595d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 -- 172.21.15.46:0/1362355612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559955950e90 con 0x559955954af0 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f33fff640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x55995595d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f337fe640 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f240cd2d0 tx=0x7f6f2400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7b795a1ecde8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f197fa640 1 -- 172.21.15.46:0/1362355612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2409b400 con 0x559955954af0 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f197fa640 1 -- 172.21.15.46:0/1362355612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2409baa0 con 0x559955954af0 2024-09-12T02:53:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.021+0000 7f6f197fa640 1 -- 172.21.15.46:0/1362355612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240afc80 con 0x559955954af0 2024-09-12T02:53:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1362355612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 msgr2=0x559955957560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f240cd2d0 tx=0x7f6f2400fb00 comp rx=0 tx=0).stop 2024-09-12T02:53:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1362355612 shutdown_connections 2024-09-12T02:53:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55995595fcb0 0x55995595d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559955955410 0x559955957e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/1362355612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955957560 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1362355612 >> 172.21.15.46:0/1362355612 conn(0x559955951060 msgr2=0x55995595ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1362355612 shutdown_connections 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- 172.21.15.46:0/1362355612 wait complete. 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 -- start start 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.022+0000 7f6f3c9db740 1 Processor -- start 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955955410 0x559955963e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x559955964390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ad440 con 0x559955954af0 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ae8b0 con 0x559955955410 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341adb10 con 0x55995595fcb0 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f3c9db740 10 client.0 _ll_register_callbacks cb 0x7f6f28009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57484/0 (socket says 172.21.15.46:57484) 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 -- 172.21.15.46:0/3994823009 learned_addr learned my addr 172.21.15.46:0/3994823009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f33fff640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x559955964390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955955410 0x559955963e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 -- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 msgr2=0x559955964390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x559955964390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 -- 172.21.15.46:0/3994823009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955955410 msgr2=0x559955963e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955955410 0x559955963e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 -- 172.21.15.46:0/3994823009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599559ddd30 con 0x559955954af0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f337fe640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f24097f00 tx=0x7f6f240a4800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf19f94d8817cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.023+0000 7f6f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2409b870 con 0x559955954af0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6f240afc80 con 0x559955954af0 2024-09-12T02:53:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2409baa0 con 0x559955954af0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f2409fe50 con 0x559955954af0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599559def60 con 0x559955954af0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2409baa0 con 0x559955954af0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6f2408b1b0 con 0x559955954af0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 20 client.15585 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5599559df180 0x5599559e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 4 client.15585 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599558c0d70 con 0x5599559df180 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.024+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5599559df180 0x5599559e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.025+0000 7f6f32ffd640 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5599559df180 0x5599559e1570 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e1c4de79018176 server_cookie=b7df3d97630f1e17 in_seq=0 out_seq=0 2024-09-12T02:53:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.025+0000 7f6f12ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f12ffd640 1 -- 172.21.15.46:0/3994823009 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599558c0d70 con 0x5599559df180 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f12ffd640 10 client.15585 handle_command_reply: tid=0 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 client.15585 shutdown 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 msgr2=0x559955963910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f24097f00 tx=0x7f6f240a4800 comp rx=0 tx=0).stop 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 shutdown_connections 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5599559df180 0x5599559e1570 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55995595fcb0 0x559955964390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559955955410 0x559955963e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 --2- 172.21.15.46:0/3994823009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559955954af0 0x559955963910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 >> 172.21.15.46:0/3994823009 conn(0x559955951060 msgr2=0x55995594f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f12ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 shutdown_connections 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 1 -- 172.21.15.46:0/3994823009 wait complete. 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.032+0000 7f6f3c9db740 20 client.15585 trim_cache size 0 max 16384 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:53:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 msgr2=0x7f6f1c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 0x7f6f1c0a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005fd0 tx=0x7f6f2c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 msgr2=0x7f6f34198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c920 tx=0x7f6f2400cde0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 shutdown_connections 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6f1c0a2da0 0x7f6f1c0a5260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f3410c0b0 0x7f6f3419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f34103600 0x7f6f34198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 --2- 172.21.15.46:0/692924970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f34102ce0 0x7f6f34198220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 >> 172.21.15.46:0/692924970 conn(0x7f6f34077990 msgr2=0x7f6f340fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 shutdown_connections 2024-09-12T02:53:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:17.033+0000 7f6f39c47640 1 -- 172.21.15.46:0/692924970 wait complete. 2024-09-12T02:53:17.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:17.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:53:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:53:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:53:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:18 smithi046 ceph-mon[30951]: pgmap v96: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:18 smithi145 ceph-mon[34674]: pgmap v96: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:18 smithi196 ceph-mon[34587]: pgmap v96: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:20 smithi046 ceph-mon[30951]: pgmap v97: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:20.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:20 smithi145 ceph-mon[34674]: pgmap v97: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:20 smithi196 ceph-mon[34587]: pgmap v97: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:22.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:22 smithi196 ceph-mon[34587]: pgmap v98: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:22 smithi046 ceph-mon[30951]: pgmap v98: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:22 smithi145 ceph-mon[34674]: pgmap v98: 129 pgs: 129 active+clean; 2.8 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:24 smithi046 ceph-mon[30951]: pgmap v99: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:24 smithi145 ceph-mon[34674]: pgmap v99: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:24 smithi196 ceph-mon[34587]: pgmap v99: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:26 smithi046 ceph-mon[30951]: pgmap v100: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:26 smithi145 ceph-mon[34674]: pgmap v100: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:26 smithi196 ceph-mon[34587]: pgmap v100: 129 pgs: 129 active+clean; 3.0 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:28 smithi046 ceph-mon[30951]: pgmap v101: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:28 smithi145 ceph-mon[34674]: pgmap v101: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:28 smithi196 ceph-mon[34587]: pgmap v101: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 -- 172.21.15.46:0/4101885893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d81069c0 msgr2=0x7f05d8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 --2- 172.21.15.46:0/4101885893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d81069c0 0x7f05d8106da0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009930 tx=0x7f05c802a420 comp rx=0 tx=0).stop 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 -- 172.21.15.46:0/4101885893 shutdown_connections 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 --2- 172.21.15.46:0/4101885893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d8101350 0x7f05d810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 --2- 172.21.15.46:0/4101885893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 0x7f05d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 --2- 172.21.15.46:0/4101885893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d81069c0 0x7f05d8106da0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 -- 172.21.15.46:0/4101885893 >> 172.21.15.46:0/4101885893 conn(0x7f05d8077990 msgr2=0x7f05d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 -- 172.21.15.46:0/4101885893 shutdown_connections 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.128+0000 7f05de41e640 1 -- 172.21.15.46:0/4101885893 wait complete. 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 Processor -- start 2024-09-12T02:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 -- start start 2024-09-12T02:53:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 0x7f05d819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d81069c0 0x7f05d819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8113f10 con 0x7f05d8101350 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8113d90 con 0x7f05d81009b0 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05de41e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8114090 con 0x7f05d81069c0 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48112/0 (socket says 172.21.15.46:48112) 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05cffff640 1 -- 172.21.15.46:0/1643051130 learned_addr learned my addr 172.21.15.46:0/1643051130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.129+0000 7f05d7fff640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 0x7f05d819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d81069c0 0x7f05d819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 -- 172.21.15.46:0/1643051130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d81069c0 msgr2=0x7f05d819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d81069c0 0x7f05d819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 -- 172.21.15.46:0/1643051130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 msgr2=0x7f05d819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 0x7f05d819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 -- 172.21.15.46:0/1643051130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d819f550 con 0x7f05d8101350 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05cffff640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d950 tx=0x7f05c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05d5ffb640 1 -- 172.21.15.46:0/1643051130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0017070 con 0x7f05d8101350 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d819f840 con 0x7f05d8101350 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05d5ffb640 1 -- 172.21.15.46:0/1643051130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c0009c30 con 0x7f05d8101350 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05d5ffb640 1 -- 172.21.15.46:0/1643051130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000f620 con 0x7f05d8101350 2024-09-12T02:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.130+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d81a70d0 con 0x7f05d8101350 2024-09-12T02:53:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 Processor -- start 2024-09-12T02:53:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05d5ffb640 1 -- 172.21.15.46:0/1643051130 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f05c00109a0 con 0x7f05d8101350 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 -- start start 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cbded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ccb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8c222f0 con 0x55b1c8cbe410 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8bca270 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.133+0000 7f05e128d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8c1e2b0 con 0x55b1c8cc8cb0 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48114/0 (socket says 172.21.15.46:48114) 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 -- 172.21.15.46:0/3621795627 learned_addr learned my addr 172.21.15.46:0/3621795627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cbded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05cffff640 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ccb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 -- 172.21.15.46:0/3621795627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 msgr2=0x55b1c8ccb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ccb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 -- 172.21.15.46:0/3621795627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 msgr2=0x55b1c8cbded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d5ffb640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 0x7f05c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cbded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 -- 172.21.15.46:0/3621795627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c8bc9340 con 0x55b1c8cbe410 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 0x7f05c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d5ffb640 1 -- 172.21.15.46:0/1643051130 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f05c00cf330 con 0x7f05d8101350 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f05c802a940 tx=0x7f05c8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910083352d4c6c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.134+0000 7f05ce7fc640 1 -- 172.21.15.46:0/3621795627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8040070 con 0x55b1c8cbe410 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05ce7fc640 1 -- 172.21.15.46:0/3621795627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c80316a0 con 0x55b1c8cbe410 2024-09-12T02:53:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05ce7fc640 1 -- 172.21.15.46:0/3621795627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8035040 con 0x55b1c8cbe410 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 -- 172.21.15.46:0/3621795627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 msgr2=0x55b1c8cc8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 0x7f05c40a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05d0005fd0 tx=0x7f05d0004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f05c802a940 tx=0x7f05c8005c00 comp rx=0 tx=0).stop 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 -- 172.21.15.46:0/3621795627 shutdown_connections 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ccb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc8770 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 --2- 172.21.15.46:0/3621795627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cbded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.135+0000 7f05e128d740 1 -- 172.21.15.46:0/3621795627 >> 172.21.15.46:0/3621795627 conn(0x55b1c8cba060 msgr2=0x55b1c8cba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.136+0000 7f05e128d740 1 -- 172.21.15.46:0/3621795627 shutdown_connections 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.136+0000 7f05e128d740 1 -- 172.21.15.46:0/3621795627 wait complete. 2024-09-12T02:53:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.136+0000 7f05e128d740 1 -- start start 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.136+0000 7f05e128d740 1 Processor -- start 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cc65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8ccb880 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8bab290 con 0x55b1c8cbe410 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1c8ccba00 con 0x55b1c8cc8cb0 2024-09-12T02:53:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05e128d740 10 client.0 _ll_register_callbacks cb 0x55b1c8bdc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48122/0 (socket says 172.21.15.46:48122) 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 -- 172.21.15.46:0/708277806 learned_addr learned my addr 172.21.15.46:0/708277806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cc65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05cffff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 -- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 msgr2=0x55b1c8cc65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cc65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 -- 172.21.15.46:0/708277806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 msgr2=0x55b1c8cc6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 -- 172.21.15.46:0/708277806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c8935210 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05cffff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:53:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05d7fff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05c8031a60 tx=0x7f05c8032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93781af316bba88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.137+0000 7f05b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c804b020 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f05c80055c0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c80316a0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05c8056050 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1c8a01620 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c80316a0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f05c803c800 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 20 client.15597 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8cd8a10 0x55b1c8cdae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.138+0000 7f05e128d740 4 client.15597 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.139+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c8c2fa20 con 0x55b1c8cd8a10 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.139+0000 7f05cffff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8cd8a10 0x55b1c8cdae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.139+0000 7f05b77fe640 10 client.15597 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.139+0000 7f05cffff640 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8cd8a10 0x55b1c8cdae00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9aed7d224feab0 server_cookie=9a382efcd44b75de in_seq=0 out_seq=0 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05b77fe640 1 -- 172.21.15.46:0/708277806 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1c8c2fa20 con 0x55b1c8cd8a10 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05b77fe640 10 client.15597 handle_command_reply: tid=0 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 client.15597 shutdown 2024-09-12T02:53:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 msgr2=0x55b1c8cc5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05c8031a60 tx=0x7f05c8032ef0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 shutdown_connections 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8cd8a10 0x55b1c8cdae00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cc65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbe410 0x55b1c8cc6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 --2- 172.21.15.46:0/708277806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc5b20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 >> 172.21.15.46:0/708277806 conn(0x55b1c8cba060 msgr2=0x55b1c8cbfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05b77fe640 0 client.15597 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 shutdown_connections 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 1 -- 172.21.15.46:0/708277806 wait complete. 2024-09-12T02:53:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.140+0000 7f05e128d740 20 client.15597 trim_cache size 0 max 16384 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 Processor -- start 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 -- start start 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ce0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81b3920 con 0x55b1c8cbe410 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81b11c0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05e128d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81af010 con 0x55b1c8cc8cb0 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48130/0 (socket says 172.21.15.46:48130) 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.157+0000 7f05cffff640 1 -- 172.21.15.46:0/2684715385 learned_addr learned my addr 172.21.15.46:0/2684715385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ce0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05d7fff640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 -- 172.21.15.46:0/2684715385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 msgr2=0x55b1c8ce0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ce0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 -- 172.21.15.46:0/2684715385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 msgr2=0x55b1c8cc63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 -- 172.21.15.46:0/2684715385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c8cb9e90 con 0x55b1c8cbe410 2024-09-12T02:53:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05d7fff640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cffff640 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f05c0097280 tx=0x7f05c00d1840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583567a7537c0dff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cdffb640 1 -- 172.21.15.46:0/2684715385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0020040 con 0x55b1c8cbe410 2024-09-12T02:53:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cdffb640 1 -- 172.21.15.46:0/2684715385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c0009de0 con 0x55b1c8cbe410 2024-09-12T02:53:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.158+0000 7f05cdffb640 1 -- 172.21.15.46:0/2684715385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0097c10 con 0x55b1c8cbe410 2024-09-12T02:53:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 -- 172.21.15.46:0/2684715385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 msgr2=0x55b1c8ce0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f05c0097280 tx=0x7f05c00d1840 comp rx=0 tx=0).stop 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 -- 172.21.15.46:0/2684715385 shutdown_connections 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8ce0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ce0400 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 --2- 172.21.15.46:0/2684715385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cc63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 -- 172.21.15.46:0/2684715385 >> 172.21.15.46:0/2684715385 conn(0x55b1c8cba060 msgr2=0x55b1c8cc4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 -- 172.21.15.46:0/2684715385 shutdown_connections 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.159+0000 7f05e128d740 1 -- 172.21.15.46:0/2684715385 wait complete. 2024-09-12T02:53:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 -- start start 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 Processor -- start 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81af680 con 0x55b1c8cbe410 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81b0af0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81afd50 con 0x55b1c8cc8cb0 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05e128d740 10 client.0 _ll_register_callbacks cb 0x7f05d0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05d7fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37878/0 (socket says 172.21.15.46:37878) 2024-09-12T02:53:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05cffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48138/0 (socket says 172.21.15.46:48138) 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.160+0000 7f05d7fff640 1 -- 172.21.15.46:0/3104524710 learned_addr learned my addr 172.21.15.46:0/3104524710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 -- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 msgr2=0x55b1c8cccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 -- 172.21.15.46:0/3104524710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 msgr2=0x55b1c8ccc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 -- 172.21.15.46:0/3104524710 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c8ccd1e0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05dc9bb640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05cffff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05d7fff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f05c8033000 tx=0x7f05c8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58e82fae83b14fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c805f060 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c0009de0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f05c8004240 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.161+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05c8066050 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 10 client.25220 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1c8d43280 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c0009de0 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f05c8004770 con 0x55b1c8cbdaf0 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 10 client.25220 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 20 client.25220 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8d43970 0x55b1c8d45d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 4 client.25220 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:53:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1c8c29d70 con 0x55b1c8d43970 2024-09-12T02:53:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.162+0000 7f05cffff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8d43970 0x55b1c8d45d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.163+0000 7f05b77fe640 10 client.25220 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.163+0000 7f05cffff640 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8d43970 0x55b1c8d45d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f48bb5d62e7b554 server_cookie=77171d3e40a731f8 in_seq=0 out_seq=0 2024-09-12T02:53:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05b77fe640 1 -- 172.21.15.46:0/3104524710 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1c8c29d70 con 0x55b1c8d43970 2024-09-12T02:53:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05b77fe640 10 client.25220 handle_command_reply: tid=0 2024-09-12T02:53:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 client.25220 shutdown 2024-09-12T02:53:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 msgr2=0x55b1c8cd1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f05c8033000 tx=0x7f05c8005fd0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 shutdown_connections 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b1c8d43970 0x55b1c8d45d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1c8cc8cb0 0x55b1c8cccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b1c8cbe410 0x55b1c8ccc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 --2- 172.21.15.46:0/3104524710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b1c8cbdaf0 0x55b1c8cd1290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05b77fe640 0 client.25220 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 >> 172.21.15.46:0/3104524710 conn(0x55b1c8cba060 msgr2=0x55b1c8cbf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 shutdown_connections 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 1 -- 172.21.15.46:0/3104524710 wait complete. 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.170+0000 7f05e128d740 20 client.25220 trim_cache size 0 max 16384 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:53:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:53:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 msgr2=0x7f05c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 0x7f05c40a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05d0005fd0 tx=0x7f05d0004a40 comp rx=0 tx=0).stop 2024-09-12T02:53:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 msgr2=0x7f05d819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d950 tx=0x7f05c000de10 comp rx=0 tx=0).stop 2024-09-12T02:53:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 shutdown_connections 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f05c40a2da0 0x7f05c40a5260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d81069c0 0x7f05d819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05d8101350 0x7f05d819aaa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 --2- 172.21.15.46:0/1643051130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d81009b0 0x7f05d819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 >> 172.21.15.46:0/1643051130 conn(0x7f05d8077990 msgr2=0x7f05d8103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 shutdown_connections 2024-09-12T02:53:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:30.171+0000 7f05de41e640 1 -- 172.21.15.46:0/1643051130 wait complete. 2024-09-12T02:53:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:30 smithi046 ceph-mon[30951]: pgmap v102: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:30 smithi145 ceph-mon[34674]: pgmap v102: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:30 smithi196 ceph-mon[34587]: pgmap v102: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:31 smithi145 ceph-mon[34674]: pgmap v103: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:31 smithi196 ceph-mon[34587]: pgmap v103: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:31.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:31 smithi046 ceph-mon[30951]: pgmap v103: 129 pgs: 129 active+clean; 3.1 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:33.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:33 smithi145 ceph-mon[34674]: pgmap v104: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:33.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:33 smithi196 ceph-mon[34587]: pgmap v104: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:34.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:33 smithi046 ceph-mon[30951]: pgmap v104: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:35.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:35 smithi145 ceph-mon[34674]: pgmap v105: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:35.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:35 smithi196 ceph-mon[34587]: pgmap v105: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:36.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:35 smithi046 ceph-mon[30951]: pgmap v105: 129 pgs: 129 active+clean; 3.2 GiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:37.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:37 smithi145 ceph-mon[34674]: pgmap v106: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:37.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:37 smithi196 ceph-mon[34587]: pgmap v106: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:38.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:37 smithi046 ceph-mon[30951]: pgmap v106: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:39 smithi145 ceph-mon[34674]: pgmap v107: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:39.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:39 smithi196 ceph-mon[34587]: pgmap v107: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:40.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:39 smithi046 ceph-mon[30951]: pgmap v107: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:41.834 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:41 smithi046 ceph-mon[30951]: pgmap v108: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:41 smithi145 ceph-mon[34674]: pgmap v108: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:41.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:41 smithi196 ceph-mon[34587]: pgmap v108: 129 pgs: 129 active+clean; 3.4 GiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.263+0000 7f52c7e65640 1 -- 172.21.15.46:0/3774398525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01069c0 msgr2=0x7f52c0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.263+0000 7f52c7e65640 1 --2- 172.21.15.46:0/3774398525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01069c0 0x7f52c0106da0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009930 tx=0x7f52bc02a420 comp rx=0 tx=0).stop 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 -- 172.21.15.46:0/3774398525 shutdown_connections 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 --2- 172.21.15.46:0/3774398525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c0101350 0x7f52c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 --2- 172.21.15.46:0/3774398525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c01009b0 0x7f52c0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 --2- 172.21.15.46:0/3774398525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01069c0 0x7f52c0106da0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 -- 172.21.15.46:0/3774398525 >> 172.21.15.46:0/3774398525 conn(0x7f52c0077990 msgr2=0x7f52c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 -- 172.21.15.46:0/3774398525 shutdown_connections 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 -- 172.21.15.46:0/3774398525 wait complete. 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 Processor -- start 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.264+0000 7f52c7e65640 1 -- start start 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c0101350 0x7f52c0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 0x7f52c019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0110d00 con 0x7f52c01009b0 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0110b80 con 0x7f52c01069c0 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c7e65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0110e80 con 0x7f52c0101350 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58742/0 (socket says 172.21.15.46:58742) 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 -- 172.21.15.46:0/517550746 learned_addr learned my addr 172.21.15.46:0/517550746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5400640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c0101350 0x7f52c0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c6402640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 0x7f52c019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 -- 172.21.15.46:0/517550746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c0101350 msgr2=0x7f52c0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c0101350 0x7f52c0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 -- 172.21.15.46:0/517550746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 msgr2=0x7f52c019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 0x7f52c019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c5c01640 1 -- 172.21.15.46:0/517550746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c019d2d0 con 0x7f52c01009b0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.265+0000 7f52c6402640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 0x7f52c019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52c5c01640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52b6ffd640 1 -- 172.21.15.46:0/517550746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc03f070 con 0x7f52c01009b0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52b6ffd640 1 -- 172.21.15.46:0/517550746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52bc031b50 con 0x7f52c01009b0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c019d560 con 0x7f52c01009b0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52b6ffd640 1 -- 172.21.15.46:0/517550746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035030 con 0x7f52c01009b0 2024-09-12T02:53:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.266+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01a4e50 con 0x7f52c01009b0 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52b6ffd640 1 -- 172.21.15.46:0/517550746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f52bc042070 con 0x7f52c01009b0 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 Processor -- start 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 -- start start 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c6866eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68679770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c6867c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c685d32f0 con 0x558c6866f410 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c6857b270 con 0x558c68679cb0 2024-09-12T02:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52cac01740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c685cf2b0 con 0x558c6866eaf0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52c5c01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68679770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52c6402640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c6866eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52c5c01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68679770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58752/0 (socket says 172.21.15.46:58752) 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52c5c01640 1 -- 172.21.15.46:0/832822194 learned_addr learned my addr 172.21.15.46:0/832822194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.269+0000 7f52c5400640 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c6867c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52b6ffd640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 0x7f52ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 -- 172.21.15.46:0/832822194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 msgr2=0x558c6867c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c6867c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 -- 172.21.15.46:0/832822194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 msgr2=0x558c68679770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68679770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 -- 172.21.15.46:0/832822194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6857a340 con 0x558c6866eaf0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52b6ffd640 1 -- 172.21.15.46:0/517550746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f52bc0efa00 con 0x7f52c01009b0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 0x7f52ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52c6402640 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c6866eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005fd0 tx=0x7f52b8011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce9750f9264cf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52abfff640 1 -- 172.21.15.46:0/832822194 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8020070 con 0x558c6866eaf0 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52abfff640 1 -- 172.21.15.46:0/832822194 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b8003e30 con 0x558c6866eaf0 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.270+0000 7f52abfff640 1 -- 172.21.15.46:0/832822194 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b801b790 con 0x558c6866eaf0 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 -- 172.21.15.46:0/832822194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 msgr2=0x558c6866eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c6866eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005fd0 tx=0x7f52b8011bf0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52c6402640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 0x7f52ac0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52b801d000 tx=0x7f52b801bec0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 -- 172.21.15.46:0/832822194 shutdown_connections 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c6867c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68679770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 --2- 172.21.15.46:0/832822194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c6866eed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 -- 172.21.15.46:0/832822194 >> 172.21.15.46:0/832822194 conn(0x558c6866b060 msgr2=0x558c6866b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 -- 172.21.15.46:0/832822194 shutdown_connections 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.271+0000 7f52cac01740 1 -- 172.21.15.46:0/832822194 wait complete. 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 -- start start 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 Processor -- start 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c68676b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866f410 0x558c68677090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c6867c880 con 0x558c68679cb0 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c6855c290 con 0x558c6866eaf0 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c6867ca00 con 0x558c6866f410 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52cac01740 10 client.0 _ll_register_callbacks cb 0x558c6858d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52c6402640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52c6402640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58764/0 (socket says 172.21.15.46:58764) 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52c6402640 1 -- 172.21.15.46:0/3066814037 learned_addr learned my addr 172.21.15.46:0/3066814037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52c5400640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866f410 0x558c68677090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.272+0000 7f52c5c01640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c68676b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 -- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866f410 msgr2=0x558c68677090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866f410 0x558c68677090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 -- 172.21.15.46:0/3066814037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 msgr2=0x558c68676b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c68676b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 -- 172.21.15.46:0/3066814037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c682e6210 con 0x558c68679cb0 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52c6402640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f52b801e040 tx=0x7f52b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ddf97e34d7eb3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8020040 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b8003e30 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f52b8004050 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52b803e050 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52cac01740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.273+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c683b2620 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b8003e30 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f52b800f9a0 con 0x558c68679cb0 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 20 client.15615 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c68689bf0 0x558c6868bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 4 client.15615 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:53:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c685e0a20 con 0x558c68689bf0 2024-09-12T02:53:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52c5c01640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c68689bf0 0x558c6868bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.274+0000 7f52c5c01640 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c68689bf0 0x558c6868bfe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc417c37ed99fcc server_cookie=ddb2b1e086b0c04a in_seq=0 out_seq=0 2024-09-12T02:53:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52a97fa640 10 client.15615 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52a97fa640 1 -- 172.21.15.46:0/3066814037 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c685e0a20 con 0x558c68689bf0 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52a97fa640 10 client.15615 handle_command_reply: tid=0 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 client.15615 shutdown 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 msgr2=0x558c686775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f52b801e040 tx=0x7f52b8016000 comp rx=0 tx=0).stop 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 shutdown_connections 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c68689bf0 0x558c6868bfe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c68679cb0 0x558c686775d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866f410 0x558c68677090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52a97fa640 0 client.15615 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 --2- 172.21.15.46:0/3066814037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c68676b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.275+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 >> 172.21.15.46:0/3066814037 conn(0x558c6866b060 msgr2=0x558c68670d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.276+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 shutdown_connections 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.276+0000 7f52cac01740 1 -- 172.21.15.46:0/3066814037 wait complete. 2024-09-12T02:53:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.276+0000 7f52cac01740 20 client.15615 trim_cache size 0 max 16384 2024-09-12T02:53:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.296+0000 7f52cac01740 1 Processor -- start 2024-09-12T02:53:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.296+0000 7f52cac01740 1 -- start start 2024-09-12T02:53:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c68677400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c68691da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01b16b0 con 0x558c6866f410 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01aef50 con 0x558c68679cb0 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52cac01740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01acda0 con 0x558c6866eaf0 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58772/0 (socket says 172.21.15.46:58772) 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c5400640 1 -- 172.21.15.46:0/2112226985 learned_addr learned my addr 172.21.15.46:0/2112226985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c68691da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c5c01640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c68677400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 -- 172.21.15.46:0/2112226985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 msgr2=0x558c68677400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c68677400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 -- 172.21.15.46:0/2112226985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 msgr2=0x558c68691860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 -- 172.21.15.46:0/2112226985 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6868c350 con 0x558c68679cb0 2024-09-12T02:53:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c5400640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.297+0000 7f52c6402640 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c68691da0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52b8011c30 tx=0x7f52b8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcbd288f5a8ad6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52ab7fe640 1 -- 172.21.15.46:0/2112226985 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8003e30 con 0x558c68679cb0 2024-09-12T02:53:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52ab7fe640 1 -- 172.21.15.46:0/2112226985 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b8035070 con 0x558c68679cb0 2024-09-12T02:53:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52ab7fe640 1 -- 172.21.15.46:0/2112226985 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80113f0 con 0x558c68679cb0 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 -- 172.21.15.46:0/2112226985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 msgr2=0x558c68691da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c68691da0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52b8011c30 tx=0x7f52b8009500 comp rx=0 tx=0).stop 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 -- 172.21.15.46:0/2112226985 shutdown_connections 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c68679cb0 0x558c68691da0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c68691860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 --2- 172.21.15.46:0/2112226985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6866eaf0 0x558c68677400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 -- 172.21.15.46:0/2112226985 >> 172.21.15.46:0/2112226985 conn(0x558c6866b060 msgr2=0x558c68678bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 -- 172.21.15.46:0/2112226985 shutdown_connections 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.298+0000 7f52cac01740 1 -- 172.21.15.46:0/2112226985 wait complete. 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 -- start start 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 Processor -- start 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c6867d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c68679cb0 0x558c686f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ad410 con 0x558c6866f410 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ae880 con 0x558c6866eaf0 2024-09-12T02:53:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01adae0 con 0x558c68679cb0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52cac01740 10 client.0 _ll_register_callbacks cb 0x7f52b801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58782/0 (socket says 172.21.15.46:58782) 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52c5400640 1 -- 172.21.15.46:0/4233751357 learned_addr learned my addr 172.21.15.46:0/4233751357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.299+0000 7f52c5c01640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c6867d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c6402640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c68679cb0 0x558c686f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 -- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c68679cb0 msgr2=0x558c686f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c68679cb0 0x558c686f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 -- 172.21.15.46:0/4233751357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 msgr2=0x558c6867d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c6867d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 -- 172.21.15.46:0/4233751357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c686f2d60 con 0x558c6866f410 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52c5400640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f52b000bb90 tx=0x7f52b00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a50bd4bcc36b3be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0005220 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b8035070 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f52b00053c0 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52b0028050 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52cac01740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.300+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c686f4150 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b8035070 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f52b001d6a0 con 0x558c6866f410 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 20 client.15621 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c686f4910 0x558c686f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 4 client.15621 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c685dad70 con 0x558c686f4910 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52c5c01640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c686f4910 0x558c686f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52a97fa640 10 client.15621 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.301+0000 7f52c5c01640 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c686f4910 0x558c686f6d30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d610d67c8103aa server_cookie=9e91a3cc5ced6ccf in_seq=0 out_seq=0 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52a97fa640 1 -- 172.21.15.46:0/4233751357 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c685dad70 con 0x558c686f4910 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52a97fa640 10 client.15621 handle_command_reply: tid=0 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 client.15621 shutdown 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 msgr2=0x558c6867d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f52b000bb90 tx=0x7f52b00039c0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 shutdown_connections 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558c686f4910 0x558c686f6d30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c68679cb0 0x558c686f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52a97fa640 0 client.15621 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c6866f410 0x558c6867d780 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 --2- 172.21.15.46:0/4233751357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c6866eaf0 0x558c6867d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 >> 172.21.15.46:0/4233751357 conn(0x558c6866b060 msgr2=0x558c686699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 shutdown_connections 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 1 -- 172.21.15.46:0/4233751357 wait complete. 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.309+0000 7f52cac01740 20 client.15621 trim_cache size 0 max 16384 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 msgr2=0x7f52ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 0x7f52ac0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52b801d000 tx=0x7f52b801bec0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 msgr2=0x7f52c01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc003b90 comp rx=0 tx=0).stop 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 shutdown_connections 2024-09-12T02:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f52ac0a2e20 0x7f52ac0a52e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52c01069c0 0x7f52c019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c0101350 0x7f52c0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 --2- 172.21.15.46:0/517550746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c01009b0 0x7f52c01982e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 >> 172.21.15.46:0/517550746 conn(0x7f52c0077990 msgr2=0x7f52c0069940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 shutdown_connections 2024-09-12T02:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:43.310+0000 7f52c7e65640 1 -- 172.21.15.46:0/517550746 wait complete. 2024-09-12T02:53:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:43 smithi145 ceph-mon[34674]: pgmap v109: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:43 smithi196 ceph-mon[34587]: pgmap v109: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:44.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:43 smithi046 ceph-mon[30951]: pgmap v109: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:45 smithi145 ceph-mon[34674]: pgmap v110: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:45 smithi196 ceph-mon[34587]: pgmap v110: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:45 smithi046 ceph-mon[30951]: pgmap v110: 129 pgs: 129 active+clean; 3.5 GiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:53:47.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:47 smithi145 ceph-mon[34674]: pgmap v111: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:47 smithi196 ceph-mon[34587]: pgmap v111: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:48.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:47 smithi046 ceph-mon[30951]: pgmap v111: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:49.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:49 smithi145 ceph-mon[34674]: pgmap v112: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:49.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:49 smithi196 ceph-mon[34587]: pgmap v112: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:50.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:49 smithi046 ceph-mon[30951]: pgmap v112: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:51 smithi145 ceph-mon[34674]: pgmap v113: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:51 smithi196 ceph-mon[34587]: pgmap v113: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:52.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:51 smithi046 ceph-mon[30951]: pgmap v113: 129 pgs: 129 active+clean; 3.6 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:53.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:53 smithi145 ceph-mon[34674]: pgmap v114: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:53 smithi196 ceph-mon[34587]: pgmap v114: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:54.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:53 smithi046 ceph-mon[30951]: pgmap v114: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:53:55.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:55 smithi145 ceph-mon[34674]: pgmap v115: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:55.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:55 smithi196 ceph-mon[34587]: pgmap v115: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:55 smithi046 ceph-mon[30951]: pgmap v115: 129 pgs: 129 active+clean; 3.8 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 -- 172.21.15.46:0/1382316818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 msgr2=0x7ff4a010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/1382316818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a010e9a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff498009cd0 tx=0x7ff49802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 -- 172.21.15.46:0/1382316818 shutdown_connections 2024-09-12T02:53:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/1382316818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a010e9a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/1382316818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0101ad0 0x7ff4a0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/1382316818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0107b70 0x7ff4a0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 -- 172.21.15.46:0/1382316818 >> 172.21.15.46:0/1382316818 conn(0x7ff4a00fd7d0 msgr2=0x7ff4a00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 -- 172.21.15.46:0/1382316818 shutdown_connections 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.402+0000 7ff4a54a6640 1 -- 172.21.15.46:0/1382316818 wait complete. 2024-09-12T02:53:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.403+0000 7ff4a54a6640 1 Processor -- start 2024-09-12T02:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.403+0000 7ff4a54a6640 1 -- start start 2024-09-12T02:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.403+0000 7ff4a54a6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 0x7ff4a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff4a54a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff4a54a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 0x7ff4a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff4a54a6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0111dd0 con 0x7ff4a0102470 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff4a54a6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0111c50 con 0x7ff4a0101ad0 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff4a54a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0111f50 con 0x7ff4a0107b70 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33306/0 (socket says 172.21.15.46:33306) 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 -- 172.21.15.46:0/2901821328 learned_addr learned my addr 172.21.15.46:0/2901821328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49effd640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 0x7ff4a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 0x7ff4a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 -- 172.21.15.46:0/2901821328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 msgr2=0x7ff4a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 0x7ff4a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 -- 172.21.15.46:0/2901821328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 msgr2=0x7ff4a00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 0x7ff4a00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 -- 172.21.15.46:0/2901821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a01a6740 con 0x7ff4a0102470 2024-09-12T02:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 0x7ff4a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49effd640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 0x7ff4a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d770 tx=0x7ff49400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.404+0000 7ff487fff640 1 -- 172.21.15.46:0/2901821328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494017070 con 0x7ff4a0102470 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.405+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a01a6a30 con 0x7ff4a0102470 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.405+0000 7ff487fff640 1 -- 172.21.15.46:0/2901821328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4940054d0 con 0x7ff4a0102470 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.405+0000 7ff487fff640 1 -- 172.21.15.46:0/2901821328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494004cf0 con 0x7ff4a0102470 2024-09-12T02:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.405+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a6f30 con 0x7ff4a0102470 2024-09-12T02:53:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff487fff640 1 -- 172.21.15.46:0/2901821328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff494004e90 con 0x7ff4a0102470 2024-09-12T02:53:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 Processor -- start 2024-09-12T02:53:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 -- start start 2024-09-12T02:53:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 0x564055dbfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 0x564055dca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564055d242f0 con 0x564055dcacb0 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564055ccc270 con 0x564055dc0410 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.407+0000 7ff4a823a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564055d202b0 con 0x564055dbfaf0 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33308/0 (socket says 172.21.15.46:33308) 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 -- 172.21.15.46:0/1749960001 learned_addr learned my addr 172.21.15.46:0/1749960001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49effd640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 0x564055dca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 0x564055dbfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff487fff640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 0x7ff4880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 -- 172.21.15.46:0/1749960001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 msgr2=0x564055dbfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 0x564055dbfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 -- 172.21.15.46:0/1749960001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 msgr2=0x564055dca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 0x564055dca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 -- 172.21.15.46:0/1749960001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564055ccb340 con 0x564055dcacb0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 0x564055dbfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49effd640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 0x564055dca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 0x7ff4880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff487fff640 1 -- 172.21.15.46:0/2901821328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff4940cf220 con 0x7ff4a0102470 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.408+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cc580 tx=0x7ff494066010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1856a7cfe0ec4cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4857fa640 1 -- 172.21.15.46:0/1749960001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49409af20 con 0x564055dcacb0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4857fa640 1 -- 172.21.15.46:0/1749960001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494009c30 con 0x564055dcacb0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4857fa640 1 -- 172.21.15.46:0/1749960001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494005180 con 0x564055dcacb0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 0x7ff4880a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff498009a60 tx=0x7ff49803e040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 -- 172.21.15.46:0/1749960001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 msgr2=0x564055dcd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cc580 tx=0x7ff494066010 comp rx=0 tx=0).stop 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 -- 172.21.15.46:0/1749960001 shutdown_connections 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dcd0a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dc0410 0x564055dca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1749960001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dbfaf0 0x564055dbfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 -- 172.21.15.46:0/1749960001 >> 172.21.15.46:0/1749960001 conn(0x564055dbc060 msgr2=0x564055dbc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 -- 172.21.15.46:0/1749960001 shutdown_connections 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.409+0000 7ff4a823a740 1 -- 172.21.15.46:0/1749960001 wait complete. 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 -- start start 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 Processor -- start 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564055dcd820 con 0x564055dcacb0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564055cad290 con 0x564055dbfaf0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564055dcd9a0 con 0x564055dc0410 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff4a823a740 10 client.0 _ll_register_callbacks cb 0x564055cde140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff49f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff49f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33310/0 (socket says 172.21.15.46:33310) 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff49f7fe640 1 -- 172.21.15.46:0/3973690302 learned_addr learned my addr 172.21.15.46:0/3973690302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.410+0000 7ff49effd640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 -- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 msgr2=0x564055dc8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 -- 172.21.15.46:0/3973690302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 msgr2=0x564055dc7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 -- 172.21.15.46:0/3973690302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564055a37210 con 0x564055dcacb0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49effd640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff498003bf0 tx=0x7ff498005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c971ddf5bdab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff482ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498004510 con 0x564055dcacb0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff4980046b0 con 0x564055dcacb0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564055d31a20 con 0x564055dcacb0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4980305a0 con 0x564055dcacb0 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff4a823a740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564055b03620 con 0x564055dcacb0 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.411+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564055d31a20 con 0x564055dcacb0 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff49804e020 con 0x564055dcacb0 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 20 client.15633 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055ddab50 0x564055ddcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 4 client.15633 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564055d2bd70 con 0x564055ddab50 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.412+0000 7ff49effd640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055ddab50 0x564055ddcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff49effd640 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055ddab50 0x564055ddcf40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6bf4a7cb692bf1 server_cookie=15046084d0a84ca8 in_seq=0 out_seq=0 2024-09-12T02:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff482ff5640 10 client.15633 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff482ff5640 1 -- 172.21.15.46:0/3973690302 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564055d2bd70 con 0x564055ddab50 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff482ff5640 10 client.15633 handle_command_reply: tid=0 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 client.15633 shutdown 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 msgr2=0x564055dc85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff498003bf0 tx=0x7ff498005c50 comp rx=0 tx=0).stop 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 shutdown_connections 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055ddab50 0x564055ddcf40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dcacb0 0x564055dc85a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3973690302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.413+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 >> 172.21.15.46:0/3973690302 conn(0x564055dbc060 msgr2=0x564055dc1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.414+0000 7ff482ff5640 0 client.15633 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.414+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 shutdown_connections 2024-09-12T02:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.414+0000 7ff4a823a740 1 -- 172.21.15.46:0/3973690302 wait complete. 2024-09-12T02:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.414+0000 7ff4a823a740 20 client.15633 trim_cache size 0 max 16384 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.429+0000 7ff4a823a740 1 Processor -- start 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.429+0000 7ff4a823a740 1 -- start start 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055de2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01b38d0 con 0x564055dbc060 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01b1170 con 0x564055dbfaf0 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff4a823a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01aefc0 con 0x564055dc0410 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33316/0 (socket says 172.21.15.46:33316) 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 -- 172.21.15.46:0/1152253513 learned_addr learned my addr 172.21.15.46:0/1152253513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055de2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 -- 172.21.15.46:0/1152253513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 msgr2=0x564055de2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055de2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 -- 172.21.15.46:0/1152253513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 msgr2=0x564055dc9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 -- 172.21.15.46:0/1152253513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564055dbbe90 con 0x564055dbc060 2024-09-12T02:53:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.430+0000 7ff49effd640 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff49000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7f8e9040d43dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff484ff9640 1 -- 172.21.15.46:0/1152253513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490004220 con 0x564055dbc060 2024-09-12T02:53:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff484ff9640 1 -- 172.21.15.46:0/1152253513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff490011e50 con 0x564055dbc060 2024-09-12T02:53:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff484ff9640 1 -- 172.21.15.46:0/1152253513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900047a0 con 0x564055dbc060 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 -- 172.21.15.46:0/1152253513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 msgr2=0x564055dc8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff49000b390 comp rx=0 tx=0).stop 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 -- 172.21.15.46:0/1152253513 shutdown_connections 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055de2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 --2- 172.21.15.46:0/1152253513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc8380 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.431+0000 7ff4a823a740 1 -- 172.21.15.46:0/1152253513 >> 172.21.15.46:0/1152253513 conn(0x564055dbc9e0 msgr2=0x564055dc9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- 172.21.15.46:0/1152253513 shutdown_connections 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- 172.21.15.46:0/1152253513 wait complete. 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- start start 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 Processor -- start 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01af630 con 0x564055dbc060 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01b0aa0 con 0x564055dbfaf0 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01afd00 con 0x564055dc0410 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff4a823a740 10 client.0 _ll_register_callbacks cb 0x7ff498009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff49effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff49effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33332/0 (socket says 172.21.15.46:33332) 2024-09-12T02:53:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.432+0000 7ff49effd640 1 -- 172.21.15.46:0/3450241887 learned_addr learned my addr 172.21.15.46:0/3450241887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49f7fe640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 -- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 msgr2=0x564055dc44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 -- 172.21.15.46:0/3450241887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 msgr2=0x564055dc3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 -- 172.21.15.46:0/3450241887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564055dd1c60 con 0x564055dbc060 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff49effd640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff490005c60 tx=0x7ff4900050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be3df1b2b09eeb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff482ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490029070 con 0x564055dbc060 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff490010b30 con 0x564055dbc060 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff490011e50 con 0x564055dbc060 2024-09-12T02:53:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff49003b050 con 0x564055dbc060 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff4a823a740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.433+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564055dd2f70 con 0x564055dbc060 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff490011e50 con 0x564055dbc060 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff49002d590 con 0x564055dbc060 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 20 client.15639 populate_metadata read hostname 'smithi046' 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055dcacb0 0x564055e47830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 4 client.15639 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564055ca8900 con 0x564055dcacb0 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.434+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055dcacb0 0x564055e47830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.435+0000 7ff49e7fc640 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055dcacb0 0x564055e47830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d1ea3e478e11a9 server_cookie=60df742e590ccae8 in_seq=0 out_seq=0 2024-09-12T02:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.435+0000 7ff482ff5640 10 client.15639 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff482ff5640 1 -- 172.21.15.46:0/3450241887 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564055ca8900 con 0x564055dcacb0 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff482ff5640 10 client.15639 handle_command_reply: tid=0 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 client.15639 shutdown 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 msgr2=0x564055dc57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff490005c60 tx=0x7ff4900050a0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 shutdown_connections 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564055dcacb0 0x564055e47830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564055dc0410 0x564055dc44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564055dbfaf0 0x564055dc3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff482ff5640 0 client.15639 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 --2- 172.21.15.46:0/3450241887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564055dbc060 0x564055dc57c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 >> 172.21.15.46:0/3450241887 conn(0x564055dbc9e0 msgr2=0x564055dc1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 shutdown_connections 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 1 -- 172.21.15.46:0/3450241887 wait complete. 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.443+0000 7ff4a823a740 20 client.15639 trim_cache size 0 max 16384 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 msgr2=0x7ff4880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 0x7ff4880a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff498009a60 tx=0x7ff49803e040 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 msgr2=0x7ff4a0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d770 tx=0x7ff49400dc30 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 shutdown_connections 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff4880a2da0 0x7ff4880a5260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4a0107b70 0x7ff4a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0102470 0x7ff4a0074a30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 --2- 172.21.15.46:0/2901821328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4a0101ad0 0x7ff4a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 >> 172.21.15.46:0/2901821328 conn(0x7ff4a00fd7d0 msgr2=0x7ff4a00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 shutdown_connections 2024-09-12T02:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:53:56.444+0000 7ff4a54a6640 1 -- 172.21.15.46:0/2901821328 wait complete. 2024-09-12T02:53:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:57 smithi145 ceph-mon[34674]: pgmap v116: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:57 smithi196 ceph-mon[34587]: pgmap v116: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:57 smithi046 ceph-mon[30951]: pgmap v116: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:53:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:53:59 smithi145 ceph-mon[34674]: pgmap v117: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:53:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:53:59 smithi196 ceph-mon[34587]: pgmap v117: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:53:59 smithi046 ceph-mon[30951]: pgmap v117: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:01 smithi145 ceph-mon[34674]: pgmap v118: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:01.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:01 smithi196 ceph-mon[34587]: pgmap v118: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:02.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:01 smithi046 ceph-mon[30951]: pgmap v118: 129 pgs: 129 active+clean; 3.9 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:03 smithi145 ceph-mon[34674]: pgmap v119: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:03 smithi196 ceph-mon[34587]: pgmap v119: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:04.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:03 smithi046 ceph-mon[30951]: pgmap v119: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:05.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:05 smithi145 ceph-mon[34674]: pgmap v120: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:54:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:05 smithi196 ceph-mon[34587]: pgmap v120: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:54:06.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:05 smithi046 ceph-mon[30951]: pgmap v120: 129 pgs: 129 active+clean; 4.0 GiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:54:07.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:07 smithi145 ceph-mon[34674]: pgmap v121: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:07 smithi196 ceph-mon[34587]: pgmap v121: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:07 smithi046 ceph-mon[30951]: pgmap v121: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:08.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:08 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:08.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:08 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:09.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:08 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 -- 172.21.15.46:0/2588365868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 msgr2=0x7f957810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 --2- 172.21.15.46:0/2588365868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f957810d880 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9570009cd0 tx=0x7f957002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 -- 172.21.15.46:0/2588365868 shutdown_connections 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 --2- 172.21.15.46:0/2588365868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f957810d880 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 --2- 172.21.15.46:0/2588365868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781009b0 0x7f9578100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 --2- 172.21.15.46:0/2588365868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781069c0 0x7f9578106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 -- 172.21.15.46:0/2588365868 >> 172.21.15.46:0/2588365868 conn(0x7f9578077990 msgr2=0x7f95780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 -- 172.21.15.46:0/2588365868 shutdown_connections 2024-09-12T02:54:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.539+0000 7f957d027640 1 -- 172.21.15.46:0/2588365868 wait complete. 2024-09-12T02:54:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 Processor -- start 2024-09-12T02:54:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 -- start start 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781009b0 0x7f9578197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781069c0 0x7f957819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9578110d00 con 0x7f9578101350 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9578110b80 con 0x7f95781009b0 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957d027640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9578110e80 con 0x7f95781069c0 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43430/0 (socket says 172.21.15.46:43430) 2024-09-12T02:54:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957659c640 1 -- 172.21.15.46:0/247250345 learned_addr learned my addr 172.21.15.46:0/247250345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f9576d9d640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781009b0 0x7f9578197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.540+0000 7f957759e640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781069c0 0x7f957819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 -- 172.21.15.46:0/247250345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781069c0 msgr2=0x7f957819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781069c0 0x7f957819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 -- 172.21.15.46:0/247250345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781009b0 msgr2=0x7f9578197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781009b0 0x7f9578197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 -- 172.21.15.46:0/247250345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819c100 con 0x7f9578101350 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957659c640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f956c00d950 tx=0x7f956c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f955ffff640 1 -- 172.21.15.46:0/247250345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c017030 con 0x7f9578101350 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819c3f0 con 0x7f9578101350 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f955ffff640 1 -- 172.21.15.46:0/247250345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956c004020 con 0x7f9578101350 2024-09-12T02:54:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f955ffff640 1 -- 172.21.15.46:0/247250345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0047a0 con 0x7f9578101350 2024-09-12T02:54:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.541+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781a3c80 con 0x7f9578101350 2024-09-12T02:54:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.544+0000 7f955ffff640 1 -- 172.21.15.46:0/247250345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f956c0041d0 con 0x7f9578101350 2024-09-12T02:54:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.544+0000 7f957fdc0740 1 Processor -- start 2024-09-12T02:54:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.544+0000 7f957fdc0740 1 -- start start 2024-09-12T02:54:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.544+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d2000f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957fdc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d2000502f0 con 0x55d2000ec410 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957fdc0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d1ffff8270 con 0x55d2000ebaf0 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957fdc0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d20004c2b0 con 0x55d2000f6cb0 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f955ffff640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 0x7f95600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43438/0 (socket says 172.21.15.46:43438) 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 -- 172.21.15.46:0/875484975 learned_addr learned my addr 172.21.15.46:0/875484975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957759e640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 0x7f95600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f955ffff640 1 -- 172.21.15.46:0/247250345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f956c0cf1a0 con 0x7f9578101350 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957659c640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d2000f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957759e640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 -- 172.21.15.46:0/875484975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 msgr2=0x55d2000f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d2000f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 -- 172.21.15.46:0/875484975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 msgr2=0x55d2000ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 -- 172.21.15.46:0/875484975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ffff7340 con 0x55d2000ec410 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957759e640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f9576d9d640 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f956800bc50 tx=0x7f95680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f49d334126baee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.545+0000 7f957759e640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 0x7f95600a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9570004af0 tx=0x7f9570009360 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f955d7fa640 1 -- 172.21.15.46:0/875484975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956801c480 con 0x55d2000ec410 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f955d7fa640 1 -- 172.21.15.46:0/875484975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956801cad0 con 0x55d2000ec410 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f955d7fa640 1 -- 172.21.15.46:0/875484975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568011b70 con 0x55d2000ec410 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 -- 172.21.15.46:0/875484975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 msgr2=0x55d2000f6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f956800bc50 tx=0x7f95680107a0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 -- 172.21.15.46:0/875484975 shutdown_connections 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d2000f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f6770 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 --2- 172.21.15.46:0/875484975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 -- 172.21.15.46:0/875484975 >> 172.21.15.46:0/875484975 conn(0x55d2000e8060 msgr2=0x55d2000e8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 -- 172.21.15.46:0/875484975 shutdown_connections 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.546+0000 7f957fdc0740 1 -- 172.21.15.46:0/875484975 wait complete. 2024-09-12T02:54:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 -- start start 2024-09-12T02:54:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 Processor -- start 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000ebaf0 0x55d2000f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000f6cb0 0x55d2000f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d2000fba90 con 0x55d2000ec410 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d1fffd9290 con 0x55d2000f6cb0 2024-09-12T02:54:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d2000fbc10 con 0x55d2000ebaf0 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957fdc0740 10 client.0 _ll_register_callbacks cb 0x55d20000a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43440/0 (socket says 172.21.15.46:43440) 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957659c640 1 -- 172.21.15.46:0/1694185124 learned_addr learned my addr 172.21.15.46:0/1694185124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.547+0000 7f957759e640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000f6cb0 0x55d2000f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000ebaf0 0x55d2000f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 -- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000ebaf0 msgr2=0x55d2000f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000ebaf0 0x55d2000f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 -- 172.21.15.46:0/1694185124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000f6cb0 msgr2=0x55d2000f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000f6cb0 0x55d2000f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 -- 172.21.15.46:0/1694185124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ffd63210 con 0x55d2000ec410 2024-09-12T02:54:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957659c640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ccb90 tx=0x7f956c09ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954135d10365e60e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f955aff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b4060 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f956c0979d0 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956801cad0 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f956c0a3fd0 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957fdc0740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1ffe2f620 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.548+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956801cad0 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f956c0ccce0 con 0x55d2000ec410 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 20 client.15651 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200108d80 0x55d20010b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 4 client.15651 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d20005da20 con 0x55d200108d80 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200108d80 0x55d20010b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.549+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200108d80 0x55d20010b170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6dccd71b9301da server_cookie=7582c89be47856e in_seq=0 out_seq=0 2024-09-12T02:54:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f955aff5640 10 client.15651 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f955aff5640 1 -- 172.21.15.46:0/1694185124 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d20005da20 con 0x55d200108d80 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f955aff5640 10 client.15651 handle_command_reply: tid=0 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 client.15651 shutdown 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 msgr2=0x55d2000f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ccb90 tx=0x7f956c09ad80 comp rx=0 tx=0).stop 2024-09-12T02:54:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 shutdown_connections 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200108d80 0x55d20010b170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000f6cb0 0x55d2000f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d2000f4050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1694185124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000ebaf0 0x55d2000f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 >> 172.21.15.46:0/1694185124 conn(0x55d2000e8060 msgr2=0x55d2000eddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.550+0000 7f955aff5640 0 client.15651 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.551+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 shutdown_connections 2024-09-12T02:54:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.551+0000 7f957fdc0740 1 -- 172.21.15.46:0/1694185124 wait complete. 2024-09-12T02:54:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.551+0000 7f957fdc0740 20 client.15651 trim_cache size 0 max 16384 2024-09-12T02:54:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.566+0000 7f957fdc0740 1 Processor -- start 2024-09-12T02:54:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.566+0000 7f957fdc0740 1 -- start start 2024-09-12T02:54:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.566+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.566+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200110f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957fdc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95781af450 con 0x55d2000ec410 2024-09-12T02:54:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957fdc0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95781accf0 con 0x55d2000ebaf0 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957fdc0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95781aab40 con 0x55d2000f6cb0 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43442/0 (socket says 172.21.15.46:43442) 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 -- 172.21.15.46:0/1826651688 learned_addr learned my addr 172.21.15.46:0/1826651688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957759e640 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200110f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 -- 172.21.15.46:0/1826651688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 msgr2=0x55d200110f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200110f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 -- 172.21.15.46:0/1826651688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 msgr2=0x55d2000f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 -- 172.21.15.46:0/1826651688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2000e7e90 con 0x55d2000ec410 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f957659c640 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f956c097d30 tx=0x7f956c09adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10143cd1012e722 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.567+0000 7f955cff9640 1 -- 172.21.15.46:0/1826651688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b4040 con 0x55d2000ec410 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f955cff9640 1 -- 172.21.15.46:0/1826651688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956c07b050 con 0x55d2000ec410 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f955cff9640 1 -- 172.21.15.46:0/1826651688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0979d0 con 0x55d2000ec410 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 -- 172.21.15.46:0/1826651688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 msgr2=0x55d200110a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f956c097d30 tx=0x7f956c09adf0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 -- 172.21.15.46:0/1826651688 shutdown_connections 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ec410 0x55d200110a40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1826651688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ebaf0 0x55d2000f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 -- 172.21.15.46:0/1826651688 >> 172.21.15.46:0/1826651688 conn(0x55d2000e8060 msgr2=0x55d2000f5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 -- 172.21.15.46:0/1826651688 shutdown_connections 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.568+0000 7f957fdc0740 1 -- 172.21.15.46:0/1826651688 wait complete. 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 -- start start 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 Processor -- start 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ec410 0x55d200100e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200101620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95781ab1b0 con 0x55d2000ebaf0 2024-09-12T02:54:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95781ac620 con 0x55d2000ec410 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95781ab880 con 0x55d2000f6cb0 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957fdc0740 10 client.0 _ll_register_callbacks cb 0x7f956c004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43458/0 (socket says 172.21.15.46:43458) 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 -- 172.21.15.46:0/1585123544 learned_addr learned my addr 172.21.15.46:0/1585123544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957659c640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ec410 0x55d200100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f957759e640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 -- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 msgr2=0x55d200101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200101620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 -- 172.21.15.46:0/1585123544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ec410 msgr2=0x55d200100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ec410 0x55d200100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.569+0000 7f9576d9d640 1 -- 172.21.15.46:0/1585123544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2000f0cc0 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f9576d9d640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f956801e010 tx=0x7f9568011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695b86664a3cbfcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568005450 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f95680055f0 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956c07b050 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f956800b850 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f957fdc0740 10 client.15657 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2001761c0 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956c07b050 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f956802f070 con 0x55d2000ebaf0 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f957fdc0740 10 client.15657 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.570+0000 7f957fdc0740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957fdc0740 20 client.15657 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200176800 0x55d200178c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957fdc0740 4 client.15657 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d200057d70 con 0x55d200176800 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957659c640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200176800 0x55d200178c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f955aff5640 10 client.15657 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.571+0000 7f957659c640 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200176800 0x55d200178c20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f60827f14ae7033 server_cookie=13dd245b6787aafa in_seq=0 out_seq=0 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f955aff5640 1 -- 172.21.15.46:0/1585123544 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d200057d70 con 0x55d200176800 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f955aff5640 10 client.15657 handle_command_reply: tid=0 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 client.15657 shutdown 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 msgr2=0x55d20016f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f956801e010 tx=0x7f9568011730 comp rx=0 tx=0).stop 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 shutdown_connections 2024-09-12T02:54:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d200176800 0x55d200178c20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2000f6cb0 0x55d200101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2000ec410 0x55d200100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f955aff5640 0 client.15657 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 --2- 172.21.15.46:0/1585123544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2000ebaf0 0x55d20016f4b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 >> 172.21.15.46:0/1585123544 conn(0x55d2000e8060 msgr2=0x55d2000eddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 shutdown_connections 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 1 -- 172.21.15.46:0/1585123544 wait complete. 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.579+0000 7f957fdc0740 20 client.15657 trim_cache size 0 max 16384 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:54:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:54:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:54:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:54:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 msgr2=0x7f95600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 0x7f95600a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9570004af0 tx=0x7f9570009360 comp rx=0 tx=0).stop 2024-09-12T02:54:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 msgr2=0x7f9578197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f956c00d950 tx=0x7f956c00de10 comp rx=0 tx=0).stop 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 shutdown_connections 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f95600a2da0 0x7f95600a5260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95781069c0 0x7f957819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9578101350 0x7f9578197650 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 --2- 172.21.15.46:0/247250345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95781009b0 0x7f9578197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 >> 172.21.15.46:0/247250345 conn(0x7f9578077990 msgr2=0x7f95780fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 shutdown_connections 2024-09-12T02:54:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:09.580+0000 7f957d027640 1 -- 172.21.15.46:0/247250345 wait complete. 2024-09-12T02:54:09.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:09 smithi145 ceph-mon[34674]: pgmap v122: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:09 smithi196 ceph-mon[34587]: pgmap v122: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:10.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:09 smithi046 ceph-mon[30951]: pgmap v122: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:11.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:11 smithi145 ceph-mon[34674]: pgmap v123: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:11.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:11 smithi196 ceph-mon[34587]: pgmap v123: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:12.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:11 smithi046 ceph-mon[30951]: pgmap v123: 129 pgs: 129 active+clean; 4.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:54:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:13 smithi145 ceph-mon[34674]: pgmap v124: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:13 smithi196 ceph-mon[34587]: pgmap v124: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:14.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:13 smithi046 ceph-mon[30951]: pgmap v124: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:54:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:15 smithi145 ceph-mon[34674]: pgmap v125: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:15 smithi196 ceph-mon[34587]: pgmap v125: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:15 smithi046 ceph-mon[30951]: pgmap v125: 129 pgs: 129 active+clean; 4.3 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:17 smithi196 ceph-mon[34587]: pgmap v126: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:54:18.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:17 smithi046 ceph-mon[30951]: pgmap v126: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:54:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:17 smithi145 ceph-mon[34674]: pgmap v126: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:54:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:19 smithi196 ceph-mon[34587]: pgmap v127: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:20.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:19 smithi046 ceph-mon[30951]: pgmap v127: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:19 smithi145 ceph-mon[34674]: pgmap v127: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:21.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:21 smithi196 ceph-mon[34587]: pgmap v128: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:21 smithi046 ceph-mon[30951]: pgmap v128: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:22.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:21 smithi145 ceph-mon[34674]: pgmap v128: 129 pgs: 129 active+clean; 4.4 GiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1575739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff01035a0 msgr2=0x7f0ff010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1575739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff01035a0 0x7f0ff010ba80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009900 tx=0x7f0fdc02a480 comp rx=0 tx=0).stop 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1575739868 shutdown_connections 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1575739868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff010c0d0 0x7f0ff010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1575739868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff01035a0 0x7f0ff010ba80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.675+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1575739868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 0x7f0ff0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1575739868 >> 172.21.15.46:0/1575739868 conn(0x7f0ff0077990 msgr2=0x7f0ff00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1575739868 shutdown_connections 2024-09-12T02:54:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1575739868 wait complete. 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 Processor -- start 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 -- start start 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 0x7f0ff019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff01035a0 0x7f0ff019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.676+0000 7f0ff6a8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff6a8d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0111d60 con 0x7f0ff010c0d0 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff6a8d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0111be0 con 0x7f0ff0102c80 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff6a8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0111ee0 con 0x7f0ff01035a0 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57238/0 (socket says 172.21.15.46:57238) 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 -- 172.21.15.46:0/1624087086 learned_addr learned my addr 172.21.15.46:0/1624087086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff4829640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 0x7f0ff019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0feffff640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff01035a0 0x7f0ff019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 -- 172.21.15.46:0/1624087086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff01035a0 msgr2=0x7f0ff019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff01035a0 0x7f0ff019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 -- 172.21.15.46:0/1624087086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 msgr2=0x7f0ff019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 0x7f0ff019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 -- 172.21.15.46:0/1624087086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff019f4a0 con 0x7f0ff010c0d0 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0ff502a640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800eaa0 tx=0x7f0fe800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.677+0000 7f0fedffb640 1 -- 172.21.15.46:0/1624087086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800cdb0 con 0x7f0ff010c0d0 2024-09-12T02:54:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.678+0000 7f0fedffb640 1 -- 172.21.15.46:0/1624087086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe8005020 con 0x7f0ff010c0d0 2024-09-12T02:54:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.678+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff019f790 con 0x7f0ff010c0d0 2024-09-12T02:54:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.678+0000 7f0fedffb640 1 -- 172.21.15.46:0/1624087086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80116d0 con 0x7f0ff010c0d0 2024-09-12T02:54:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.678+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff01a7020 con 0x7f0ff010c0d0 2024-09-12T02:54:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.680+0000 7f0fedffb640 1 -- 172.21.15.46:0/1624087086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0fe80051d0 con 0x7f0ff010c0d0 2024-09-12T02:54:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 Processor -- start 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0fedffb640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 0x7f0fe00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 -- start start 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0feffff640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 0x7f0fe00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0fedffb640 1 -- 172.21.15.46:0/1624087086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0fe80cf890 con 0x7f0ff010c0d0 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 0x55df7e2b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df7e2092f0 con 0x55df7e2a4af0 2024-09-12T02:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55df7e1b1270 con 0x55df7e2afcb0 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff9826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df7e2052b0 con 0x55df7e2a5410 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff4829640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff4829640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57250/0 (socket says 172.21.15.46:57250) 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.681+0000 7f0ff4829640 1 -- 172.21.15.46:0/4097560372 learned_addr learned my addr 172.21.15.46:0/4097560372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff502a640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 0x55df7e2b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0feffff640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 -- 172.21.15.46:0/4097560372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 msgr2=0x55df7e2af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 -- 172.21.15.46:0/4097560372 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 msgr2=0x55df7e2b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 0x55df7e2b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 -- 172.21.15.46:0/4097560372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7e1b0340 con 0x55df7e2a4af0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0feffff640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff502a640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 0x55df7e2b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0feffff640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 0x7f0fe00a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc031440 tx=0x7f0fdc005b50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0ff4829640 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009cf0 tx=0x7f0fe4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e5bf87136e2441 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0fdaffd640 1 -- 172.21.15.46:0/4097560372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4004230 con 0x55df7e2a4af0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0fdaffd640 1 -- 172.21.15.46:0/4097560372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40043d0 con 0x55df7e2a4af0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.682+0000 7f0fdaffd640 1 -- 172.21.15.46:0/4097560372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40197e0 con 0x55df7e2a4af0 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 -- 172.21.15.46:0/4097560372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 msgr2=0x55df7e2a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009cf0 tx=0x7f0fe4011810 comp rx=0 tx=0).stop 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 -- 172.21.15.46:0/4097560372 shutdown_connections 2024-09-12T02:54:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2afcb0 0x55df7e2b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4097560372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a4ed0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 -- 172.21.15.46:0/4097560372 >> 172.21.15.46:0/4097560372 conn(0x55df7e2a1060 msgr2=0x55df7e2a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 -- 172.21.15.46:0/4097560372 shutdown_connections 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.683+0000 7f0ff9826740 1 -- 172.21.15.46:0/4097560372 wait complete. 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 -- start start 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 Processor -- start 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a4af0 0x55df7e2af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df7e2b2880 con 0x55df7e2afcb0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55df7e192290 con 0x55df7e2a4af0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df7e2b2a00 con 0x55df7e2a5410 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff9826740 10 client.0 _ll_register_callbacks cb 0x55df7e1c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57260/0 (socket says 172.21.15.46:57260) 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff502a640 1 -- 172.21.15.46:0/19273786 learned_addr learned my addr 172.21.15.46:0/19273786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0ff4829640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a4af0 0x55df7e2af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.684+0000 7f0feffff640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 -- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 msgr2=0x55df7e2aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 -- 172.21.15.46:0/19273786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a4af0 msgr2=0x55df7e2af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a4af0 0x55df7e2af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 -- 172.21.15.46:0/19273786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7df1c210 con 0x55df7e2afcb0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff502a640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80984d0 tx=0x7f0fe809b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5874612b22fa2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe801f070 con 0x55df7e2afcb0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe40043d0 con 0x55df7e2afcb0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0fe80cf450 con 0x55df7e2afcb0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fe80cf650 con 0x55df7e2afcb0 2024-09-12T02:54:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff9826740 10 client.15669 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df7dfe8620 con 0x55df7e2afcb0 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.685+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40043d0 con 0x55df7e2afcb0 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0fe800f040 con 0x55df7e2afcb0 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 10 client.15669 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 20 client.15669 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e2bf9e0 0x55df7e2c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 4 client.15669 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df7e216a20 con 0x55df7e2bf9e0 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff4829640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e2bf9e0 0x55df7e2c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0fcffff640 10 client.15669 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.686+0000 7f0ff4829640 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e2bf9e0 0x55df7e2c1dd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad16e2413e676886 server_cookie=b7cf02240ff1d43 in_seq=0 out_seq=0 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0fcffff640 1 -- 172.21.15.46:0/19273786 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df7e216a20 con 0x55df7e2bf9e0 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0fcffff640 10 client.15669 handle_command_reply: tid=0 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 client.15669 shutdown 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 msgr2=0x55df7e2ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80984d0 tx=0x7f0fe809b9e0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 shutdown_connections 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e2bf9e0 0x55df7e2c1dd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2ad1a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a5410 0x55df7e2aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 --2- 172.21.15.46:0/19273786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a4af0 0x55df7e2af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 >> 172.21.15.46:0/19273786 conn(0x55df7e2a1060 msgr2=0x55df7e2a6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0fcffff640 0 client.15669 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 shutdown_connections 2024-09-12T02:54:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.687+0000 7f0ff9826740 1 -- 172.21.15.46:0/19273786 wait complete. 2024-09-12T02:54:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.688+0000 7f0ff9826740 20 client.15669 trim_cache size 0 max 16384 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 Processor -- start 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 -- start start 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 0x55df7e2ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01b3930 con 0x55df7e2afcb0 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01b11d0 con 0x55df7e2a5410 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff9826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01af020 con 0x55df7e2a4af0 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff502a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57264/0 (socket says 172.21.15.46:57264) 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff502a640 1 -- 172.21.15.46:0/4137746173 learned_addr learned my addr 172.21.15.46:0/4137746173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff4829640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 0x55df7e2ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0feffff640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff502a640 1 -- 172.21.15.46:0/4137746173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 msgr2=0x55df7e2ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.703+0000 7f0ff502a640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 0x55df7e2ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0ff502a640 1 -- 172.21.15.46:0/4137746173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 msgr2=0x55df7e2ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0ff502a640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0ff502a640 1 -- 172.21.15.46:0/4137746173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7e2a0e90 con 0x55df7e2afcb0 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0ff4829640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 0x55df7e2ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0ff502a640 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80cd280 tx=0x7f0fe80a4840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a85196f26933fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0fda7fc640 1 -- 172.21.15.46:0/4137746173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe801f050 con 0x55df7e2afcb0 2024-09-12T02:54:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0fda7fc640 1 -- 172.21.15.46:0/4137746173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe80b4070 con 0x55df7e2afcb0 2024-09-12T02:54:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.704+0000 7f0fda7fc640 1 -- 172.21.15.46:0/4137746173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8097850 con 0x55df7e2afcb0 2024-09-12T02:54:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- 172.21.15.46:0/4137746173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 msgr2=0x55df7e2c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80cd280 tx=0x7f0fe80a4840 comp rx=0 tx=0).stop 2024-09-12T02:54:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- 172.21.15.46:0/4137746173 shutdown_connections 2024-09-12T02:54:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2afcb0 0x55df7e2c76e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 --2- 172.21.15.46:0/4137746173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2a4af0 0x55df7e2ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- 172.21.15.46:0/4137746173 >> 172.21.15.46:0/4137746173 conn(0x55df7e2a1060 msgr2=0x55df7e2a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- 172.21.15.46:0/4137746173 shutdown_connections 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- 172.21.15.46:0/4137746173 wait complete. 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 -- start start 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.705+0000 7f0ff9826740 1 Processor -- start 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2afcb0 0x55df7e2aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01af690 con 0x55df7e2a4af0 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01b0b00 con 0x55df7e2a5410 2024-09-12T02:54:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01afd60 con 0x55df7e2afcb0 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff9826740 10 client.0 _ll_register_callbacks cb 0x7f0fe8005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57268/0 (socket says 172.21.15.46:57268) 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 -- 172.21.15.46:0/356671527 learned_addr learned my addr 172.21.15.46:0/356671527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0feffff640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff502a640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2afcb0 0x55df7e2aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 -- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2afcb0 msgr2=0x55df7e2aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2afcb0 0x55df7e2aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 -- 172.21.15.46:0/356671527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 msgr2=0x55df7e2a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 -- 172.21.15.46:0/356671527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7e2b6360 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0feffff640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0ff4829640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005fd0 tx=0x7f0fe4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3613ce51c44e5479 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.706+0000 7f0fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4031060 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe80b4070 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0fe4005250 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fe4038050 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df7e32aab0 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe80b4070 con 0x55df7e2a4af0 2024-09-12T02:54:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0fe4019540 con 0x55df7e2a4af0 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 20 client.15675 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e32b040 0x55df7e32d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 4 client.15675 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.707+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df7e210d70 con 0x55df7e32b040 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.708+0000 7f0feffff640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e32b040 0x55df7e32d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.708+0000 7f0feffff640 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e32b040 0x55df7e32d430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f43445cf574668 server_cookie=f966b7a7a7b128d in_seq=0 out_seq=0 2024-09-12T02:54:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.708+0000 7f0fcffff640 10 client.15675 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0fcffff640 1 -- 172.21.15.46:0/356671527 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df7e210d70 con 0x55df7e32b040 2024-09-12T02:54:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0fcffff640 10 client.15675 handle_command_reply: tid=0 2024-09-12T02:54:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 client.15675 shutdown 2024-09-12T02:54:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 msgr2=0x55df7e2a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005fd0 tx=0x7f0fe4011810 comp rx=0 tx=0).stop 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 shutdown_connections 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55df7e32b040 0x55df7e32d430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df7e2afcb0 0x55df7e2aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df7e2a5410 0x55df7e2a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0fcffff640 0 client.15675 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 --2- 172.21.15.46:0/356671527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df7e2a4af0 0x55df7e2a9a80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 >> 172.21.15.46:0/356671527 conn(0x55df7e2a1060 msgr2=0x55df7e2ac240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 shutdown_connections 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.716+0000 7f0ff9826740 1 -- 172.21.15.46:0/356671527 wait complete. 2024-09-12T02:54:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff9826740 20 client.15675 trim_cache size 0 max 16384 2024-09-12T02:54:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:54:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:54:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:54:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 msgr2=0x7f0fe00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 0x7f0fe00a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc031440 tx=0x7f0fdc005b50 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 msgr2=0x7f0ff019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800eaa0 tx=0x7f0fe800ef60 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 shutdown_connections 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0fe00a2da0 0x7f0fe00a5260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff010c0d0 0x7f0ff019ed70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ff01035a0 0x7f0ff019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 --2- 172.21.15.46:0/1624087086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ff0102c80 0x7f0ff019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 >> 172.21.15.46:0/1624087086 conn(0x7f0ff0077990 msgr2=0x7f0ff0100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 shutdown_connections 2024-09-12T02:54:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:22.717+0000 7f0ff6a8d640 1 -- 172.21.15.46:0/1624087086 wait complete. 2024-09-12T02:54:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:23 smithi145 ceph-mon[34674]: pgmap v129: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:23.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:23 smithi196 ceph-mon[34587]: pgmap v129: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:24.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:23 smithi046 ceph-mon[30951]: pgmap v129: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:25.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:25 smithi196 ceph-mon[34587]: pgmap v130: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:26.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:25 smithi046 ceph-mon[30951]: pgmap v130: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:26.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:25 smithi145 ceph-mon[34674]: pgmap v130: 129 pgs: 129 active+clean; 4.5 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:27.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:27 smithi196 ceph-mon[34587]: pgmap v131: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:28.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:27 smithi046 ceph-mon[30951]: pgmap v131: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:28.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:27 smithi145 ceph-mon[34674]: pgmap v131: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:29 smithi196 ceph-mon[34587]: pgmap v132: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:30.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:29 smithi046 ceph-mon[30951]: pgmap v132: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:29 smithi145 ceph-mon[34674]: pgmap v132: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:31.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:31 smithi196 ceph-mon[34587]: pgmap v133: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:32.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:31 smithi046 ceph-mon[30951]: pgmap v133: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:32.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:31 smithi145 ceph-mon[34674]: pgmap v133: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:33.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:33 smithi196 ceph-mon[34587]: pgmap v134: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:34.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:33 smithi046 ceph-mon[30951]: pgmap v134: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:34.112 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:33 smithi145 ceph-mon[34674]: pgmap v134: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 -- 172.21.15.46:0/3830681270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd81067e0 msgr2=0x7fccd8106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 --2- 172.21.15.46:0/3830681270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd81067e0 0x7fccd8106bc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009930 tx=0x7fccd402a420 comp rx=0 tx=0).stop 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 -- 172.21.15.46:0/3830681270 shutdown_connections 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 --2- 172.21.15.46:0/3830681270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8101170 0x7fccd810d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 --2- 172.21.15.46:0/3830681270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81007d0 0x7fccd8100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 --2- 172.21.15.46:0/3830681270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd81067e0 0x7fccd8106bc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 -- 172.21.15.46:0/3830681270 >> 172.21.15.46:0/3830681270 conn(0x7fccd80777c0 msgr2=0x7fccd80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 -- 172.21.15.46:0/3830681270 shutdown_connections 2024-09-12T02:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.810+0000 7fccdf19f640 1 -- 172.21.15.46:0/3830681270 wait complete. 2024-09-12T02:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 Processor -- start 2024-09-12T02:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 -- start start 2024-09-12T02:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd81007d0 0x7fccd8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81067e0 0x7fccd819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110ad0 con 0x7fccd8101170 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110950 con 0x7fccd81067e0 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdf19f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110c50 con 0x7fccd81007d0 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdd99c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdd99c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35306/0 (socket says 172.21.15.46:35306) 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccdd99c640 1 -- 172.21.15.46:0/381318382 learned_addr learned my addr 172.21.15.46:0/381318382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccde19d640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd81007d0 0x7fccd8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.811+0000 7fccde99e640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81067e0 0x7fccd819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 -- 172.21.15.46:0/381318382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd81007d0 msgr2=0x7fccd8198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd81007d0 0x7fccd8198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 -- 172.21.15.46:0/381318382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81067e0 msgr2=0x7fccd819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81067e0 0x7fccd819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 -- 172.21.15.46:0/381318382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819d160 con 0x7fccd8101170 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdd99c640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fccc800d950 tx=0x7fccc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fcccf7fe640 1 -- 172.21.15.46:0/381318382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8017070 con 0x7fccd8101170 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819d450 con 0x7fccd8101170 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fcccf7fe640 1 -- 172.21.15.46:0/381318382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccc8009c30 con 0x7fccd8101170 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fcccf7fe640 1 -- 172.21.15.46:0/381318382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800f620 con 0x7fccd8101170 2024-09-12T02:54:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.812+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a4ce0 con 0x7fccd8101170 2024-09-12T02:54:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.815+0000 7fcccf7fe640 1 -- 172.21.15.46:0/381318382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fccc80109a0 con 0x7fccd8101170 2024-09-12T02:54:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.815+0000 7fcce3275740 1 Processor -- start 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 -- start start 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820d3e80 0x55d2820d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d28202d2f0 con 0x55d2820c95e0 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d281fd5270 con 0x55d2820c8cc0 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fcce3275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d2820292b0 con 0x55d2820d3e80 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fccde19d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fccde19d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35308/0 (socket says 172.21.15.46:35308) 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fccde19d640 1 -- 172.21.15.46:0/188751846 learned_addr learned my addr 172.21.15.46:0/188751846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fccde99e640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.816+0000 7fccdd99c640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820d3e80 0x55d2820d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 -- 172.21.15.46:0/188751846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820d3e80 msgr2=0x55d2820d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820d3e80 0x55d2820d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 -- 172.21.15.46:0/188751846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 msgr2=0x55d2820c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 -- 172.21.15.46:0/188751846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d281fd4340 con 0x55d2820c95e0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde99e640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fcccf7fe640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 0x7fccc40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde99e640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 0x7fccc40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fcccf7fe640 1 -- 172.21.15.46:0/381318382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fccc80cf400 con 0x7fccd8101170 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccde19d640 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fccd402a940 tx=0x7fccd4031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8847a2218cae5984 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccccbfe640 1 -- 172.21.15.46:0/188751846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4040070 con 0x55d2820c95e0 2024-09-12T02:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccccbfe640 1 -- 172.21.15.46:0/188751846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccd4031e50 con 0x55d2820c95e0 2024-09-12T02:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.817+0000 7fccccbfe640 1 -- 172.21.15.46:0/188751846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035040 con 0x55d2820c95e0 2024-09-12T02:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 -- 172.21.15.46:0/188751846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 msgr2=0x55d2820d3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fccd402a940 tx=0x7fccd4031ba0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fccde99e640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 0x7fccc40a51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccd0005fd0 tx=0x7fccd0009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 -- 172.21.15.46:0/188751846 shutdown_connections 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820d3e80 0x55d2820d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820d3940 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 --2- 172.21.15.46:0/188751846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 -- 172.21.15.46:0/188751846 >> 172.21.15.46:0/188751846 conn(0x55d2820c5230 msgr2=0x55d2820c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 -- 172.21.15.46:0/188751846 shutdown_connections 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.818+0000 7fcce3275740 1 -- 172.21.15.46:0/188751846 wait complete. 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 -- start start 2024-09-12T02:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 Processor -- start 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d2820d6a50 con 0x55d2820d3e80 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d281fb6290 con 0x55d2820c8cc0 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d2820d6bd0 con 0x55d2820c95e0 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fcce3275740 10 client.0 _ll_register_callbacks cb 0x55d281fe7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccdd99c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35310/0 (socket says 172.21.15.46:35310) 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 -- 172.21.15.46:0/1789418205 learned_addr learned my addr 172.21.15.46:0/1789418205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde19d640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 -- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 msgr2=0x55d2820d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 -- 172.21.15.46:0/1789418205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 msgr2=0x55d2820d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.819+0000 7fccde99e640 1 -- 172.21.15.46:0/1789418205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d281d40210 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccde99e640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccd0010c20 tx=0x7fccd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc6d9f2dd217f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001fc80 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fccd001c3f0 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd4031e50 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccd001fe20 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fcce3275740 10 client.15687 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d281e0c620 con 0x55d2820d3e80 2024-09-12T02:54:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccd4031e50 con 0x55d2820d3e80 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.820+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fccd0026070 con 0x55d2820d3e80 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 20 client.15687 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d2820e3df0 0x55d2820e6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 4 client.15687 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d28203aa20 con 0x55d2820e3df0 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fccde19d640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d2820e3df0 0x55d2820e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fccc1ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.821+0000 7fccde19d640 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d2820e3df0 0x55d2820e6240 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebf26a7540a9cc3 server_cookie=76bedf95ff97e3d in_seq=0 out_seq=0 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fccc1ffb640 1 -- 172.21.15.46:0/1789418205 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d28203aa20 con 0x55d2820e3df0 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fccc1ffb640 10 client.15687 handle_command_reply: tid=0 2024-09-12T02:54:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 client.15687 shutdown 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 msgr2=0x55d2820d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccd0010c20 tx=0x7fccd0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 shutdown_connections 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d2820e3df0 0x55d2820e6240 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820d3e80 0x55d2820d17a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fccc1ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 --2- 172.21.15.46:0/1789418205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820c8cc0 0x55d2820d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 >> 172.21.15.46:0/1789418205 conn(0x55d2820c5230 msgr2=0x55d2820caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 shutdown_connections 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 1 -- 172.21.15.46:0/1789418205 wait complete. 2024-09-12T02:54:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.822+0000 7fcce3275740 20 client.15687 trim_cache size 0 max 16384 2024-09-12T02:54:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 Processor -- start 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 -- start start 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c8cc0 0x55d2820d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81b1650 con 0x55d2820c95e0 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81aeef0 con 0x55d2820d3e80 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fcce3275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81acd40 con 0x55d2820c8cc0 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fccdd99c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fccdd99c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35320/0 (socket says 172.21.15.46:35320) 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fccdd99c640 1 -- 172.21.15.46:0/3334304250 learned_addr learned my addr 172.21.15.46:0/3334304250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fccde19d640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c8cc0 0x55d2820d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.837+0000 7fccde99e640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 -- 172.21.15.46:0/3334304250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c8cc0 msgr2=0x55d2820d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c8cc0 0x55d2820d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 -- 172.21.15.46:0/3334304250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 msgr2=0x55d2820ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 -- 172.21.15.46:0/3334304250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2820c5ec0 con 0x55d2820c95e0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccde99e640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccdd99c640 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fccc809b2c0 tx=0x7fccc80d2f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f59e8d75239a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccc3fff640 1 -- 172.21.15.46:0/3334304250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8020070 con 0x55d2820c95e0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccc3fff640 1 -- 172.21.15.46:0/3334304250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccc8009de0 con 0x55d2820c95e0 2024-09-12T02:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.838+0000 7fccc3fff640 1 -- 172.21.15.46:0/3334304250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc809f970 con 0x55d2820c95e0 2024-09-12T02:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- 172.21.15.46:0/3334304250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 msgr2=0x55d2820ebba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fccc809b2c0 tx=0x7fccc80d2f40 comp rx=0 tx=0).stop 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- 172.21.15.46:0/3334304250 shutdown_connections 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c95e0 0x55d2820ebba0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 --2- 172.21.15.46:0/3334304250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c8cc0 0x55d2820d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- 172.21.15.46:0/3334304250 >> 172.21.15.46:0/3334304250 conn(0x55d2820c5230 msgr2=0x55d2820d2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- 172.21.15.46:0/3334304250 shutdown_connections 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- 172.21.15.46:0/3334304250 wait complete. 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 -- start start 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.839+0000 7fcce3275740 1 Processor -- start 2024-09-12T02:54:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ad3b0 con 0x55d2820c8cc0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ae820 con 0x55d2820d3e80 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ada80 con 0x55d2820c95e0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 10 client.0 _ll_register_callbacks cb 0x7fccd0009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35336/0 (socket says 172.21.15.46:35336) 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 -- 172.21.15.46:0/874504893 learned_addr learned my addr 172.21.15.46:0/874504893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccdd99c640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde99e640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 -- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 msgr2=0x55d2820ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 -- 172.21.15.46:0/874504893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 msgr2=0x55d2820ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 -- 172.21.15.46:0/874504893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28214db70 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccdd99c640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccde19d640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55d2820c6090 tx=0x7fccd4005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb92a3d42252673 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4040040 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc8009de0 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.840+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fccd4005240 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccd4060050 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 10 client.15693 fetch_fsmap learned FSMap version 28 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d28214edc0 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccc8009de0 con 0x55d2820c8cc0 2024-09-12T02:54:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fccd40049a0 con 0x55d2820c8cc0 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 10 client.15693 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 20 client.15693 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d28214f100 0x55d2821514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 4 client.15693 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d282034d70 con 0x55d28214f100 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.841+0000 7fccdd99c640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d28214f100 0x55d2821514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.842+0000 7fccc1ffb640 10 client.15693 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.842+0000 7fccdd99c640 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d28214f100 0x55d2821514f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b45b1fe0c3bd1a4 server_cookie=42a1eb921e5b1926 in_seq=0 out_seq=0 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fccc1ffb640 1 -- 172.21.15.46:0/874504893 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d282034d70 con 0x55d28214f100 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fccc1ffb640 10 client.15693 handle_command_reply: tid=0 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 client.15693 shutdown 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 msgr2=0x55d2820cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55d2820c6090 tx=0x7fccd4005130 comp rx=0 tx=0).stop 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 shutdown_connections 2024-09-12T02:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d28214f100 0x55d2821514f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d2820d3e80 0x55d2820ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2820c95e0 0x55d2820ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fccc1ffb640 0 client.15693 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 --2- 172.21.15.46:0/874504893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2820c8cc0 0x55d2820cdc50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 >> 172.21.15.46:0/874504893 conn(0x55d2820c5230 msgr2=0x55d2820c3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 shutdown_connections 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 1 -- 172.21.15.46:0/874504893 wait complete. 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.849+0000 7fcce3275740 20 client.15693 trim_cache size 0 max 16384 2024-09-12T02:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 msgr2=0x7fccc40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 0x7fccc40a51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccd0005fd0 tx=0x7fccd0009500 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 msgr2=0x7fccd81986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fccc800d950 tx=0x7fccc800de10 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 shutdown_connections 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fccc40a2cf0 0x7fccc40a51b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccd81067e0 0x7fccd819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccd8101170 0x7fccd81986b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 --2- 172.21.15.46:0/381318382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd81007d0 0x7fccd8198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 >> 172.21.15.46:0/381318382 conn(0x7fccd80777c0 msgr2=0x7fccd8069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 shutdown_connections 2024-09-12T02:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:35.850+0000 7fccdf19f640 1 -- 172.21.15.46:0/381318382 wait complete. 2024-09-12T02:54:35.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:35 smithi196 ceph-mon[34587]: pgmap v135: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:36.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:35 smithi046 ceph-mon[30951]: pgmap v135: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:36.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:35 smithi145 ceph-mon[34674]: pgmap v135: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:37.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:37 smithi196 ceph-mon[34587]: pgmap v136: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:54:38.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:37 smithi046 ceph-mon[30951]: pgmap v136: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:54:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:37 smithi145 ceph-mon[34674]: pgmap v136: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:54:39.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:39 smithi196 ceph-mon[34587]: pgmap v137: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:40.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:39 smithi046 ceph-mon[30951]: pgmap v137: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:39 smithi145 ceph-mon[34674]: pgmap v137: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:41.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:41 smithi196 ceph-mon[34587]: pgmap v138: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:42.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:41 smithi046 ceph-mon[30951]: pgmap v138: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:41 smithi145 ceph-mon[34674]: pgmap v138: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:42.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:42 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:43.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:42 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:43.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:42 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:54:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:43 smithi196 ceph-mon[34587]: pgmap v139: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:44.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:43 smithi046 ceph-mon[30951]: pgmap v139: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:43 smithi145 ceph-mon[34674]: pgmap v139: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:45 smithi196 ceph-mon[34587]: pgmap v140: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:45 smithi046 ceph-mon[30951]: pgmap v140: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:45 smithi145 ceph-mon[34674]: pgmap v140: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:47 smithi196 ceph-mon[34587]: pgmap v141: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:48.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:47 smithi046 ceph-mon[30951]: pgmap v141: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:47 smithi145 ceph-mon[34674]: pgmap v141: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 -- 172.21.15.46:0/2573480981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0101350 msgr2=0x7f6fd010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 --2- 172.21.15.46:0/2573480981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0101350 0x7f6fd010d880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009cd0 tx=0x7f6fc802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 -- 172.21.15.46:0/2573480981 shutdown_connections 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 --2- 172.21.15.46:0/2573480981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0101350 0x7f6fd010d880 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 --2- 172.21.15.46:0/2573480981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd01009b0 0x7f6fd0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 --2- 172.21.15.46:0/2573480981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01069c0 0x7f6fd0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 -- 172.21.15.46:0/2573480981 >> 172.21.15.46:0/2573480981 conn(0x7f6fd0077990 msgr2=0x7f6fd00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 -- 172.21.15.46:0/2573480981 shutdown_connections 2024-09-12T02:54:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.948+0000 7f6fd8731640 1 -- 172.21.15.46:0/2573480981 wait complete. 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 Processor -- start 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 -- start start 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 0x7f6fd019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd0101350 0x7f6fd019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0110cb0 con 0x7f6fd01069c0 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0110b30 con 0x7f6fd01009b0 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd8731640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0110e30 con 0x7f6fd0101350 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd6cce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.949+0000 7f6fd6cce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51154/0 (socket says 172.21.15.46:51154) 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 -- 172.21.15.46:0/1072961522 learned_addr learned my addr 172.21.15.46:0/1072961522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd64cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 0x7f6fd019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd0101350 0x7f6fd019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 -- 172.21.15.46:0/1072961522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd0101350 msgr2=0x7f6fd019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd0101350 0x7f6fd019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 -- 172.21.15.46:0/1072961522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 msgr2=0x7f6fd019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 0x7f6fd019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 -- 172.21.15.46:0/1072961522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd019f3b0 con 0x7f6fd01069c0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 0x7f6fd019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd6cce640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8033040 tx=0x7f6fc8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fc77fe640 1 -- 172.21.15.46:0/1072961522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8004040 con 0x7f6fd01069c0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fc77fe640 1 -- 172.21.15.46:0/1072961522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc8004690 con 0x7f6fd01069c0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fc77fe640 1 -- 172.21.15.46:0/1072961522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8030e10 con 0x7f6fd01069c0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd019f640 con 0x7f6fd01069c0 2024-09-12T02:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.950+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd01a6f30 con 0x7f6fd01069c0 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fc77fe640 1 -- 172.21.15.46:0/1072961522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6fc80305a0 con 0x7f6fd01069c0 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 Processor -- start 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 -- start start 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f87bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f886770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e0f7e02f0 con 0x557e0f886cb0 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557e0f788270 con 0x557e0f87c410 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.953+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e0f7dc2b0 con 0x557e0f87baf0 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51164/0 (socket says 172.21.15.46:51164) 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/3440514235 learned_addr learned my addr 172.21.15.46:0/3440514235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f886770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f87bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fc77fe640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 0x7f6fbc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/3440514235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 msgr2=0x557e0f87bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f87bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/3440514235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 msgr2=0x557e0f886770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f886770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/3440514235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0f787340 con 0x557e0f886cb0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd6cce640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 0x7f6fbc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fc77fe640 1 -- 172.21.15.46:0/1072961522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6fc802e030 con 0x7f6fd01069c0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.954+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0013580 tx=0x7f6fc000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4a8059013964b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fc4ff9640 1 -- 172.21.15.46:0/3440514235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0019030 con 0x557e0f886cb0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fc4ff9640 1 -- 172.21.15.46:0/3440514235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc00054d0 con 0x557e0f886cb0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fc4ff9640 1 -- 172.21.15.46:0/3440514235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0011520 con 0x557e0f886cb0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fd6cce640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 0x7f6fbc0a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80bcd50 tx=0x7f6fc80900c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/3440514235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 msgr2=0x557e0f8890a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.955+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0013580 tx=0x7f6fc000c6b0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/3440514235 shutdown_connections 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8890a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f886770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/3440514235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f87bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/3440514235 >> 172.21.15.46:0/3440514235 conn(0x557e0f878060 msgr2=0x557e0f878440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/3440514235 shutdown_connections 2024-09-12T02:54:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/3440514235 wait complete. 2024-09-12T02:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 -- start start 2024-09-12T02:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.956+0000 7f6fdb4cc740 1 Processor -- start 2024-09-12T02:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f883bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 0x557e0f884630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e0f889880 con 0x557e0f87c410 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557e0f769290 con 0x557e0f886cb0 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e0f889a00 con 0x557e0f87baf0 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 10 client.0 _ll_register_callbacks cb 0x557e0f79a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51166/0 (socket says 172.21.15.46:51166) 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 -- 172.21.15.46:0/1641828580 learned_addr learned my addr 172.21.15.46:0/1641828580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 0x557e0f884630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 -- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 msgr2=0x557e0f883bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f883bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 -- 172.21.15.46:0/1641828580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 msgr2=0x557e0f884630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 0x557e0f884630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 -- 172.21.15.46:0/1641828580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0f4f3210 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 0x557e0f884630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc012d20 tx=0x7f6fcc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d9f9203db915e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc012110 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6fcc004050 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.957+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc00054d0 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc025050 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e0f5bf620 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc00054d0 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6fcc010430 con 0x557e0f87c410 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 20 client.15705 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f896d00 0x557e0f899130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 4 client.15705 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e0f7eda20 con 0x557e0f896d00 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.958+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f896d00 0x557e0f899130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.959+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f896d00 0x557e0f899130 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f19244f3ad9452d0 server_cookie=9ead2e97fce9ae3e in_seq=0 out_seq=0 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.959+0000 7f6fb9ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/1641828580 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e0f7eda20 con 0x557e0f896d00 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fb9ffb640 10 client.15705 handle_command_reply: tid=0 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 client.15705 shutdown 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 msgr2=0x557e0f8840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc012d20 tx=0x7f6fcc00f790 comp rx=0 tx=0).stop 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 shutdown_connections 2024-09-12T02:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f896d00 0x557e0f899130 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f886cb0 0x557e0f884630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f87c410 0x557e0f8840f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/1641828580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f883bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 >> 172.21.15.46:0/1641828580 conn(0x557e0f878060 msgr2=0x557e0f87dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fb9ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 shutdown_connections 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/1641828580 wait complete. 2024-09-12T02:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.960+0000 7f6fdb4cc740 20 client.15705 trim_cache size 0 max 16384 2024-09-12T02:54:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.976+0000 7f6fdb4cc740 1 Processor -- start 2024-09-12T02:54:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.976+0000 7f6fdb4cc740 1 -- start start 2024-09-12T02:54:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87baf0 0x557e0f884460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87c410 0x557e0f87ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b37a0 con 0x557e0f886cb0 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b1070 con 0x557e0f87baf0 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01aeec0 con 0x557e0f87c410 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51170/0 (socket says 172.21.15.46:51170) 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/4108346512 learned_addr learned my addr 172.21.15.46:0/4108346512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87baf0 0x557e0f884460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87c410 0x557e0f87ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/4108346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87c410 msgr2=0x557e0f87ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87c410 0x557e0f87ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/4108346512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87baf0 msgr2=0x557e0f884460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87baf0 0x557e0f884460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/4108346512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0f8791f0 con 0x557e0f886cb0 2024-09-12T02:54:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.977+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c5d0 tx=0x7f6fc0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5665ff61ab2c578d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fbbfff640 1 -- 172.21.15.46:0/4108346512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0019040 con 0x557e0f886cb0 2024-09-12T02:54:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fbbfff640 1 -- 172.21.15.46:0/4108346512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc0011e70 con 0x557e0f886cb0 2024-09-12T02:54:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fbbfff640 1 -- 172.21.15.46:0/4108346512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001e460 con 0x557e0f886cb0 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/4108346512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 msgr2=0x557e0f89ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c5d0 tx=0x7f6fc0015040 comp rx=0 tx=0).stop 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/4108346512 shutdown_connections 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f89ea40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87c410 0x557e0f87ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/4108346512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87baf0 0x557e0f884460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.978+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/4108346512 >> 172.21.15.46:0/4108346512 conn(0x557e0f878060 msgr2=0x557e0f872d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/4108346512 shutdown_connections 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/4108346512 wait complete. 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- start start 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 Processor -- start 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f88ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f88f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01af530 con 0x557e0f886cb0 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b09a0 con 0x557e0f87c410 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01afc00 con 0x557e0f87baf0 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fdb4cc740 10 client.0 _ll_register_callbacks cb 0x7f6fcc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fd5ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8801f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51176/0 (socket says 172.21.15.46:51176) 2024-09-12T02:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.979+0000 7f6fd64cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f88f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd5ccc640 1 -- 172.21.15.46:0/665991448 learned_addr learned my addr 172.21.15.46:0/665991448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f88ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 -- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 msgr2=0x557e0f88ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f88ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 -- 172.21.15.46:0/665991448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 msgr2=0x557e0f8801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 -- 172.21.15.46:0/665991448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0f880730 con 0x557e0f87c410 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fd64cd640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f88f200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc010850 tx=0x7f6fcc0123c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d47b7c5e9eae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc022040 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6fcc010a40 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.980+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc0011e70 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc010c40 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 10 client.25325 fetch_fsmap learned FSMap version 29 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e0f9011c0 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc0011e70 con 0x557e0f87c410 2024-09-12T02:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6fcc003920 con 0x557e0f87c410 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 10 client.25325 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 20 client.25325 populate_metadata read hostname 'smithi046' 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f901880 0x557e0f903c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 4 client.25325 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e0f7e7d70 con 0x557e0f901880 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.981+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f901880 0x557e0f903c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.982+0000 7f6fb9ffb640 10 client.25325 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.982+0000 7f6fd5ccc640 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f901880 0x557e0f903c70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22dbfccea5f5663f server_cookie=b52b109de7f66eb2 in_seq=0 out_seq=0 2024-09-12T02:54:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fb9ffb640 1 -- 172.21.15.46:0/665991448 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e0f7e7d70 con 0x557e0f901880 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fb9ffb640 10 client.25325 handle_command_reply: tid=0 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fdb4cc740 1 client.25325 shutdown 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 msgr2=0x557e0f88f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f88f200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc010850 tx=0x7f6fcc0123c0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 shutdown_connections 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.989+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x557e0f901880 0x557e0f903c70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e0f886cb0 0x557e0f8801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e0f87c410 0x557e0f88f200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 --2- 172.21.15.46:0/665991448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0f87baf0 0x557e0f88ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fb9ffb640 0 client.25325 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 >> 172.21.15.46:0/665991448 conn(0x557e0f878060 msgr2=0x557e0f882310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 shutdown_connections 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 1 -- 172.21.15.46:0/665991448 wait complete. 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fdb4cc740 20 client.25325 trim_cache size 0 max 16384 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:54:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 msgr2=0x7f6fbc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 0x7f6fbc0a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80bcd50 tx=0x7f6fc80900c0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 msgr2=0x7f6fd019ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8033040 tx=0x7f6fc8003970 comp rx=0 tx=0).stop 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 shutdown_connections 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6fbc0a2e20 0x7f6fbc0a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd01069c0 0x7f6fd019ec80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fd0101350 0x7f6fd019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 --2- 172.21.15.46:0/1072961522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fd01009b0 0x7f6fd019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 >> 172.21.15.46:0/1072961522 conn(0x7f6fd0077990 msgr2=0x7f6fd0104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 shutdown_connections 2024-09-12T02:54:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:54:48.990+0000 7f6fd8731640 1 -- 172.21.15.46:0/1072961522 wait complete. 2024-09-12T02:54:49.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:49 smithi196 ceph-mon[34587]: pgmap v142: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:50.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:49 smithi046 ceph-mon[30951]: pgmap v142: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:50.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:49 smithi145 ceph-mon[34674]: pgmap v142: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:51 smithi196 ceph-mon[34587]: pgmap v143: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:52.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:51 smithi046 ceph-mon[30951]: pgmap v143: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:52.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:51 smithi145 ceph-mon[34674]: pgmap v143: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:54:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:53 smithi145 ceph-mon[34674]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:53 smithi196 ceph-mon[34587]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:54.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:53 smithi046 ceph-mon[30951]: pgmap v144: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:55.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:55 smithi196 ceph-mon[34587]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:55 smithi046 ceph-mon[30951]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:56.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:55 smithi145 ceph-mon[34674]: pgmap v145: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:54:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:57 smithi196 ceph-mon[34587]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:57 smithi046 ceph-mon[30951]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:57 smithi145 ceph-mon[34674]: pgmap v146: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:54:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:54:59 smithi196 ceph-mon[34587]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:54:59 smithi046 ceph-mon[30951]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:00.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:54:59 smithi145 ceph-mon[34674]: pgmap v147: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:01.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:01 smithi196 ceph-mon[34587]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:02.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:01 smithi046 ceph-mon[30951]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 -- 172.21.15.46:0/1867866755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881009b0 msgr2=0x7f0d88100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/1867866755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881009b0 0x7f0d88100e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009930 tx=0x7f0d7802a420 comp rx=0 tx=0).stop 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 -- 172.21.15.46:0/1867866755 shutdown_connections 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/1867866755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d8810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/1867866755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881009b0 0x7f0d88100e10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/1867866755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881069c0 0x7f0d88106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 -- 172.21.15.46:0/1867866755 >> 172.21.15.46:0/1867866755 conn(0x7f0d88077990 msgr2=0x7f0d880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 -- 172.21.15.46:0/1867866755 shutdown_connections 2024-09-12T02:55:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.088+0000 7f0d8fd49640 1 -- 172.21.15.46:0/1867866755 wait complete. 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 Processor -- start 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 -- start start 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881009b0 0x7f0d881990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d881995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88110cb0 con 0x7f0d881069c0 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88110b30 con 0x7f0d881009b0 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8fd49640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88110e30 con 0x7f0d88101350 2024-09-12T02:55:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8e2e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8e2e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51178/0 (socket says 172.21.15.46:51178) 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8e2e6640 1 -- 172.21.15.46:0/3544833401 learned_addr learned my addr 172.21.15.46:0/3544833401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.089+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881009b0 0x7f0d881990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d881995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 -- 172.21.15.46:0/3544833401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 msgr2=0x7f0d881995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d881995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 -- 172.21.15.46:0/3544833401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881009b0 msgr2=0x7f0d881990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881009b0 0x7f0d881990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 -- 172.21.15.46:0/3544833401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881a4660 con 0x7f0d881069c0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d881995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8e2e6640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cce0 tx=0x7f0d8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d7effd640 1 -- 172.21.15.46:0/3544833401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000f450 con 0x7f0d881069c0 2024-09-12T02:55:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d7effd640 1 -- 172.21.15.46:0/3544833401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d8000faa0 con 0x7f0d881069c0 2024-09-12T02:55:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d7effd640 1 -- 172.21.15.46:0/3544833401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80018b70 con 0x7f0d881069c0 2024-09-12T02:55:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881a4830 con 0x7f0d881069c0 2024-09-12T02:55:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.090+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a4d00 con 0x7f0d881069c0 2024-09-12T02:55:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.093+0000 7f0d7effd640 1 -- 172.21.15.46:0/3544833401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0d80018d10 con 0x7f0d881069c0 2024-09-12T02:55:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.093+0000 7f0d92bba740 1 Processor -- start 2024-09-12T02:55:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.093+0000 7f0d92bba740 1 -- start start 2024-09-12T02:55:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480d410 0x563524817770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352481a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5635247712f0 con 0x56352480caf0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563524719270 con 0x56352480d410 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d92bba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56352476d2b0 con 0x563524817cb0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d7effd640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 0x7f0d740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51186/0 (socket says 172.21.15.46:51186) 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/2584682812 learned_addr learned my addr 172.21.15.46:0/2584682812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8e2e6640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 0x7f0d740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480d410 0x563524817770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352481a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d7effd640 1 -- 172.21.15.46:0/3544833401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0d800cfc00 con 0x7f0d881069c0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/2584682812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 msgr2=0x56352481a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352481a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/2584682812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480d410 msgr2=0x563524817770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480d410 0x563524817770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.094+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/2584682812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563524718340 con 0x56352480caf0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.095+0000 7f0d8e2e6640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 0x7f0d740a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d8009bac0 tx=0x7f0d8000af30 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.095+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0d78032fd0 tx=0x7f0d78005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb17074e3d68c9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.095+0000 7f0d73fff640 1 -- 172.21.15.46:0/2584682812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78035070 con 0x56352480caf0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.095+0000 7f0d73fff640 1 -- 172.21.15.46:0/2584682812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d780044d0 con 0x56352480caf0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.095+0000 7f0d73fff640 1 -- 172.21.15.46:0/2584682812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780054c0 con 0x56352480caf0 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- 172.21.15.46:0/2584682812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 msgr2=0x56352480ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0d78032fd0 tx=0x7f0d78005b50 comp rx=0 tx=0).stop 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- 172.21.15.46:0/2584682812 shutdown_connections 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352481a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480d410 0x563524817770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 --2- 172.21.15.46:0/2584682812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x56352480ced0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- 172.21.15.46:0/2584682812 >> 172.21.15.46:0/2584682812 conn(0x563524809060 msgr2=0x563524809440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- 172.21.15.46:0/2584682812 shutdown_connections 2024-09-12T02:55:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- 172.21.15.46:0/2584682812 wait complete. 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 -- start start 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.096+0000 7f0d92bba740 1 Processor -- start 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 0x5635248150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563524817cb0 0x5635248155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56352481a880 con 0x56352480caf0 2024-09-12T02:55:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5635246fa290 con 0x563524817cb0 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56352481aa00 con 0x56352480d410 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d92bba740 10 client.0 _ll_register_callbacks cb 0x56352472b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51192/0 (socket says 172.21.15.46:51192) 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/739697712 learned_addr learned my addr 172.21.15.46:0/739697712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563524817cb0 0x5635248155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 0x5635248150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 msgr2=0x5635248150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 0x5635248150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/739697712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563524817cb0 msgr2=0x5635248155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563524817cb0 0x5635248155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 -- 172.21.15.46:0/739697712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563524484210 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 0x5635248150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0d78031440 tx=0x7f0d78043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64d6d31b145e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.097+0000 7f0d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78035040 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0d780054b0 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d780044d0 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d780035f0 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563524550620 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d780044d0 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d78004880 con 0x56352480caf0 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 20 client.15723 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524827a90 0x563524829e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 4 client.15723 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.098+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56352477ea20 con 0x563524827a90 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.099+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524827a90 0x563524829e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.099+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524827a90 0x563524829e80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e201c3003b4f17f6 server_cookie=ed6035fbd9735478 in_seq=0 out_seq=0 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.099+0000 7f0d717fa640 10 client.15723 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d717fa640 1 -- 172.21.15.46:0/739697712 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56352477ea20 con 0x563524827a90 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d717fa640 10 client.15723 handle_command_reply: tid=0 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 client.15723 shutdown 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 msgr2=0x563524814b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0d78031440 tx=0x7f0d78043040 comp rx=0 tx=0).stop 2024-09-12T02:55:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 shutdown_connections 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524827a90 0x563524829e80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563524817cb0 0x5635248155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56352480d410 0x5635248150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 --2- 172.21.15.46:0/739697712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480caf0 0x563524814b70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 >> 172.21.15.46:0/739697712 conn(0x563524809060 msgr2=0x56352480edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d717fa640 0 client.15723 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 shutdown_connections 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 1 -- 172.21.15.46:0/739697712 wait complete. 2024-09-12T02:55:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.100+0000 7f0d92bba740 20 client.15723 trim_cache size 0 max 16384 2024-09-12T02:55:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:01 smithi145 ceph-mon[34674]: pgmap v148: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.114+0000 7f0d92bba740 1 Processor -- start 2024-09-12T02:55:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.114+0000 7f0d92bba740 1 -- start start 2024-09-12T02:55:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x56352480fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881b15b0 con 0x56352480d410 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881aee80 con 0x56352480caf0 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d92bba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881accd0 con 0x563524817cb0 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51198/0 (socket says 172.21.15.46:51198) 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 -- 172.21.15.46:0/386159890 learned_addr learned my addr 172.21.15.46:0/386159890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x56352480fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 -- 172.21.15.46:0/386159890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 msgr2=0x56352482cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 -- 172.21.15.46:0/386159890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 msgr2=0x56352480fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x56352480fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 -- 172.21.15.46:0/386159890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635248176c0 con 0x56352480d410 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x56352480fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.115+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x563524809ec0 tx=0x7f0d8400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f108855908bdcca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d737fe640 1 -- 172.21.15.46:0/386159890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004220 con 0x56352480d410 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d737fe640 1 -- 172.21.15.46:0/386159890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d840043c0 con 0x56352480d410 2024-09-12T02:55:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d737fe640 1 -- 172.21.15.46:0/386159890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004e60 con 0x56352480d410 2024-09-12T02:55:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 -- 172.21.15.46:0/386159890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 msgr2=0x563524815420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x563524809ec0 tx=0x7f0d8400b410 comp rx=0 tx=0).stop 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 -- 172.21.15.46:0/386159890 shutdown_connections 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524815420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 --2- 172.21.15.46:0/386159890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x56352480fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.116+0000 7f0d92bba740 1 -- 172.21.15.46:0/386159890 >> 172.21.15.46:0/386159890 conn(0x563524809060 msgr2=0x563524816bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- 172.21.15.46:0/386159890 shutdown_connections 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- 172.21.15.46:0/386159890 wait complete. 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- start start 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 Processor -- start 2024-09-12T02:55:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x563524889750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ad340 con 0x56352480d410 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ae7b0 con 0x56352480caf0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.117+0000 7f0d92bba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ada10 con 0x563524817cb0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d92bba740 10 client.0 _ll_register_callbacks cb 0x7f0d8400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51200/0 (socket says 172.21.15.46:51200) 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 -- 172.21.15.46:0/648112059 learned_addr learned my addr 172.21.15.46:0/648112059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x563524889750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 -- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 msgr2=0x56352482a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 -- 172.21.15.46:0/648112059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 msgr2=0x563524889750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x563524889750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 -- 172.21.15.46:0/648112059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56352482a8c0 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x563524889750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d8dae5640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400bc20 tx=0x7f0d84004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b86232ab8c6a052 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84029040 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f0d84004730 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.118+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d840043c0 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d8403a050 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 10 client.15729 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563524891e80 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d840043c0 con 0x56352480d410 2024-09-12T02:55:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d8402c070 con 0x56352480d410 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 10 client.15729 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 20 client.15729 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524892530 0x563524894920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 4 client.15729 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563524778d70 con 0x563524892530 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.119+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524892530 0x563524894920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.120+0000 7f0d8d2e4640 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524892530 0x563524894920 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf38db10bec4b4d server_cookie=b94611db6967472f in_seq=0 out_seq=0 2024-09-12T02:55:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.120+0000 7f0d717fa640 10 client.15729 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.128+0000 7f0d717fa640 1 -- 172.21.15.46:0/648112059 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563524778d70 con 0x563524892530 2024-09-12T02:55:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.128+0000 7f0d717fa640 10 client.15729 handle_command_reply: tid=0 2024-09-12T02:55:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.128+0000 7f0d92bba740 1 client.15729 shutdown 2024-09-12T02:55:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.128+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 msgr2=0x563524889c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.128+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400bc20 tx=0x7f0d84004d50 comp rx=0 tx=0).stop 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 shutdown_connections 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563524892530 0x563524894920 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563524817cb0 0x56352482a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d717fa640 0 client.15729 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56352480d410 0x563524889c90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 --2- 172.21.15.46:0/648112059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56352480caf0 0x563524889750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 >> 172.21.15.46:0/648112059 conn(0x563524809060 msgr2=0x5635248163d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 shutdown_connections 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 1 -- 172.21.15.46:0/648112059 wait complete. 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d92bba740 20 client.15729 trim_cache size 0 max 16384 2024-09-12T02:55:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:55:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:55:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:55:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:55:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 msgr2=0x7f0d740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 0x7f0d740a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d8009bac0 tx=0x7f0d8000af30 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 msgr2=0x7f0d881a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.129+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cce0 tx=0x7f0d8000a620 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 shutdown_connections 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0d740a2da0 0x7f0d740a5260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d881069c0 0x7f0d881a4120 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88101350 0x7f0d881995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 --2- 172.21.15.46:0/3544833401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881009b0 0x7f0d881990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 >> 172.21.15.46:0/3544833401 conn(0x7f0d88077990 msgr2=0x7f0d88103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 shutdown_connections 2024-09-12T02:55:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:02.130+0000 7f0d8fd49640 1 -- 172.21.15.46:0/3544833401 wait complete. 2024-09-12T02:55:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:03 smithi196 ceph-mon[34587]: pgmap v149: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:04.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:03 smithi046 ceph-mon[30951]: pgmap v149: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:03 smithi145 ceph-mon[34674]: pgmap v149: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:05 smithi196 ceph-mon[34587]: pgmap v150: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:55:06.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:05 smithi046 ceph-mon[30951]: pgmap v150: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:55:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:05 smithi145 ceph-mon[34674]: pgmap v150: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:55:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:07 smithi196 ceph-mon[34587]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:08.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:07 smithi046 ceph-mon[30951]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:07 smithi145 ceph-mon[34674]: pgmap v151: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:55:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:09 smithi196 ceph-mon[34587]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:10.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:09 smithi046 ceph-mon[30951]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:10.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:09 smithi145 ceph-mon[34674]: pgmap v152: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:11.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:11 smithi196 ceph-mon[34587]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:12.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:11 smithi046 ceph-mon[30951]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:12.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:11 smithi145 ceph-mon[34674]: pgmap v153: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:55:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:13 smithi196 ceph-mon[34587]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:55:14.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:13 smithi046 ceph-mon[30951]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:55:14.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:13 smithi145 ceph-mon[34674]: pgmap v154: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 39 op/s 2024-09-12T02:55:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 -- 172.21.15.46:0/1478086454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8101ad0 msgr2=0x7f63a8101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 --2- 172.21.15.46:0/1478086454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8101ad0 0x7f63a8101f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6398009930 tx=0x7f639802a420 comp rx=0 tx=0).stop 2024-09-12T02:55:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 -- 172.21.15.46:0/1478086454 shutdown_connections 2024-09-12T02:55:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 --2- 172.21.15.46:0/1478086454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8102470 0x7f63a810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 --2- 172.21.15.46:0/1478086454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8101ad0 0x7f63a8101f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 --2- 172.21.15.46:0/1478086454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8107b70 0x7f63a8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 -- 172.21.15.46:0/1478086454 >> 172.21.15.46:0/1478086454 conn(0x7f63a80fd7d0 msgr2=0x7f63a80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 -- 172.21.15.46:0/1478086454 shutdown_connections 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.232+0000 7f63afb91640 1 -- 172.21.15.46:0/1478086454 wait complete. 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 Processor -- start 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 -- start start 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8101ad0 0x7f63a81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8107b70 0x7f63a819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111dd0 con 0x7f63a8102470 2024-09-12T02:55:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111c50 con 0x7f63a8107b70 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63afb91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111f50 con 0x7f63a8101ad0 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63ad12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63ad12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50350/0 (socket says 172.21.15.46:50350) 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.233+0000 7f63ad12c640 1 -- 172.21.15.46:0/3759666064 learned_addr learned my addr 172.21.15.46:0/3759666064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad92d640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8101ad0 0x7f63a81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ae12e640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8107b70 0x7f63a819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 -- 172.21.15.46:0/3759666064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8101ad0 msgr2=0x7f63a81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8101ad0 0x7f63a81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 -- 172.21.15.46:0/3759666064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8107b70 msgr2=0x7f63a819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8107b70 0x7f63a819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 -- 172.21.15.46:0/3759666064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819d290 con 0x7f63a8102470 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63ad12c640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6398005940 tx=0x7f63980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f639effd640 1 -- 172.21.15.46:0/3759666064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639803f030 con 0x7f63a8102470 2024-09-12T02:55:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819d520 con 0x7f63a8102470 2024-09-12T02:55:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f639effd640 1 -- 172.21.15.46:0/3759666064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63980318a0 con 0x7f63a8102470 2024-09-12T02:55:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f639effd640 1 -- 172.21.15.46:0/3759666064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398035070 con 0x7f63a8102470 2024-09-12T02:55:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.234+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a4e10 con 0x7f63a8102470 2024-09-12T02:55:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.237+0000 7f639effd640 1 -- 172.21.15.46:0/3759666064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6398042030 con 0x7f63a8102470 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f639effd640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 0x7f63940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 Processor -- start 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ad92d640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 0x7f63940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f639effd640 1 -- 172.21.15.46:0/3759666064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63980eff30 con 0x7f63a8102470 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 -- start start 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc94aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc955770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc8af2f0 con 0x55e3dc955cb0 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc857270 con 0x55e3dc94aaf0 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63b2a09740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc8ab2b0 con 0x55e3dc94b410 2024-09-12T02:55:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ad92d640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 0x7f63940a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b50 tx=0x7f63a4005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ae12e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ae12e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50360/0 (socket says 172.21.15.46:50360) 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ae12e640 1 -- 172.21.15.46:0/4145765493 learned_addr learned my addr 172.21.15.46:0/4145765493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.238+0000 7f63ad92d640 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc94aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ad12c640 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc955770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 -- 172.21.15.46:0/4145765493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 msgr2=0x55e3dc955770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc955770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 -- 172.21.15.46:0/4145765493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 msgr2=0x55e3dc94aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc94aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 -- 172.21.15.46:0/4145765493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3dc856340 con 0x55e3dc955cb0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f63ae12e640 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f63a0007e50 tx=0x7f63a000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7021ab9c248e2a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f6393fff640 1 -- 172.21.15.46:0/4145765493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0014070 con 0x55e3dc955cb0 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f6393fff640 1 -- 172.21.15.46:0/4145765493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a000ee90 con 0x55e3dc955cb0 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.239+0000 7f6393fff640 1 -- 172.21.15.46:0/4145765493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0005380 con 0x55e3dc955cb0 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 -- 172.21.15.46:0/4145765493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 msgr2=0x55e3dc9580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f63a0007e50 tx=0x7f63a000e720 comp rx=0 tx=0).stop 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 -- 172.21.15.46:0/4145765493 shutdown_connections 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc9580a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc955770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 --2- 172.21.15.46:0/4145765493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc94aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 -- 172.21.15.46:0/4145765493 >> 172.21.15.46:0/4145765493 conn(0x55e3dc947060 msgr2=0x55e3dc947440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 -- 172.21.15.46:0/4145765493 shutdown_connections 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.240+0000 7f63b2a09740 1 -- 172.21.15.46:0/4145765493 wait complete. 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 -- start start 2024-09-12T02:55:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 Processor -- start 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94b410 0x55e3dc953090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 0x55e3dc9535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc958880 con 0x55e3dc94aaf0 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc838290 con 0x55e3dc94b410 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3dc958a00 con 0x55e3dc955cb0 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63b2a09740 10 client.0 _ll_register_callbacks cb 0x55e3dc869140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63ad92d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63ad92d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50370/0 (socket says 172.21.15.46:50370) 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63ad92d640 1 -- 172.21.15.46:0/2230104207 learned_addr learned my addr 172.21.15.46:0/2230104207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.241+0000 7f63ad12c640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94b410 0x55e3dc953090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ae12e640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 0x55e3dc9535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 -- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 msgr2=0x55e3dc9535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 0x55e3dc9535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 -- 172.21.15.46:0/2230104207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94b410 msgr2=0x55e3dc953090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94b410 0x55e3dc953090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 -- 172.21.15.46:0/2230104207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3dc5c2210 con 0x55e3dc94aaf0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ae12e640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 0x55e3dc9535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63ad92d640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003a70 tx=0x7f63a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aef89465ca9be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005020 con 0x55e3dc94aaf0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63a40051c0 con 0x55e3dc94aaf0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a000ee90 con 0x55e3dc94aaf0 2024-09-12T02:55:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63a40053c0 con 0x55e3dc94aaf0 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63b2a09740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.242+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3dc68e620 con 0x55e3dc94aaf0 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a000ee90 con 0x55e3dc94aaf0 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63a4026070 con 0x55e3dc94aaf0 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 20 client.15741 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc965fc0 0x55e3dc9683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 4 client.15741 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3dc8bca20 con 0x55e3dc965fc0 2024-09-12T02:55:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.243+0000 7f63ad12c640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc965fc0 0x55e3dc9683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63917fa640 10 client.15741 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63ad12c640 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc965fc0 0x55e3dc9683b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f05c60a6c04069a server_cookie=32a3a3986008605 in_seq=0 out_seq=0 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63917fa640 1 -- 172.21.15.46:0/2230104207 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3dc8bca20 con 0x55e3dc965fc0 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63917fa640 10 client.15741 handle_command_reply: tid=0 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63b2a09740 1 client.15741 shutdown 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 msgr2=0x55e3dc952b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.244+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003a70 tx=0x7f63a4004730 comp rx=0 tx=0).stop 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 shutdown_connections 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc965fc0 0x55e3dc9683b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc955cb0 0x55e3dc9535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94b410 0x55e3dc953090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63917fa640 0 client.15741 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 --2- 172.21.15.46:0/2230104207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94aaf0 0x55e3dc952b50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 >> 172.21.15.46:0/2230104207 conn(0x55e3dc947060 msgr2=0x55e3dc94cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 shutdown_connections 2024-09-12T02:55:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 1 -- 172.21.15.46:0/2230104207 wait complete. 2024-09-12T02:55:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.245+0000 7f63b2a09740 20 client.15741 trim_cache size 0 max 16384 2024-09-12T02:55:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 Processor -- start 2024-09-12T02:55:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 -- start start 2024-09-12T02:55:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94aaf0 0x55e3dc9530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc955cb0 0x55e3dc96d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81b1660 con 0x55e3dc94b410 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81aef30 con 0x55e3dc955cb0 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.268+0000 7f63b2a09740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81acd80 con 0x55e3dc94aaf0 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50378/0 (socket says 172.21.15.46:50378) 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 -- 172.21.15.46:0/650002183 learned_addr learned my addr 172.21.15.46:0/650002183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad92d640 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94aaf0 0x55e3dc9530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ae12e640 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc955cb0 0x55e3dc96d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 -- 172.21.15.46:0/650002183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94aaf0 msgr2=0x55e3dc9530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94aaf0 0x55e3dc9530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 -- 172.21.15.46:0/650002183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc955cb0 msgr2=0x55e3dc96d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc955cb0 0x55e3dc96d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 -- 172.21.15.46:0/650002183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3dc947cf0 con 0x55e3dc94b410 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63ad12c640 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f63980bbdf0 tx=0x7f639802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e42b8c2a3d6fb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63937fe640 1 -- 172.21.15.46:0/650002183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398049040 con 0x55e3dc94b410 2024-09-12T02:55:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63937fe640 1 -- 172.21.15.46:0/650002183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63980c4d60 con 0x55e3dc94b410 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.269+0000 7f63937fe640 1 -- 172.21.15.46:0/650002183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63980b8760 con 0x55e3dc94b410 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 -- 172.21.15.46:0/650002183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 msgr2=0x55e3dc9535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f63980bbdf0 tx=0x7f639802a420 comp rx=0 tx=0).stop 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 -- 172.21.15.46:0/650002183 shutdown_connections 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc955cb0 0x55e3dc96d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc94b410 0x55e3dc9535e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 --2- 172.21.15.46:0/650002183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94aaf0 0x55e3dc9530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 -- 172.21.15.46:0/650002183 >> 172.21.15.46:0/650002183 conn(0x55e3dc947060 msgr2=0x55e3dc954bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 -- 172.21.15.46:0/650002183 shutdown_connections 2024-09-12T02:55:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.270+0000 7f63b2a09740 1 -- 172.21.15.46:0/650002183 wait complete. 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 -- start start 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 Processor -- start 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc95e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc94f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc94ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ad3f0 con 0x55e3dc955cb0 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ae860 con 0x55e3dc94aaf0 2024-09-12T02:55:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81adac0 con 0x55e3dc94b410 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63b2a09740 10 client.0 _ll_register_callbacks cb 0x7f63a4004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63ae12e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc94ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63ae12e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc94ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50388/0 (socket says 172.21.15.46:50388) 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63ad92d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc95e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63ae12e640 1 -- 172.21.15.46:0/863243993 learned_addr learned my addr 172.21.15.46:0/863243993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.271+0000 7f63ad12c640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc94f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 -- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 msgr2=0x55e3dc94f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc94f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 -- 172.21.15.46:0/863243993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 msgr2=0x55e3dc94ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc94ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 -- 172.21.15.46:0/863243993 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3dc950790 con 0x55e3dc94aaf0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63ad92d640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc95e4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63a40276d0 tx=0x7f63a401f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3b237be04396a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a403a060 con 0x55e3dc94aaf0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63980c4d60 con 0x55e3dc94aaf0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63a4027b50 con 0x55e3dc94aaf0 2024-09-12T02:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63a4027d50 con 0x55e3dc94aaf0 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63b2a09740 10 client.25358 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.272+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3dc9d4eb0 con 0x55e3dc94aaf0 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63980c4d60 con 0x55e3dc94aaf0 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63a4026050 con 0x55e3dc94aaf0 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 10 client.25358 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 20 client.25358 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc9d55d0 0x55e3dc9d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 4 client.25358 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:55:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3dc8b6d70 con 0x55e3dc9d55d0 2024-09-12T02:55:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.273+0000 7f63ad12c640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc9d55d0 0x55e3dc9d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.274+0000 7f63ad12c640 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc9d55d0 0x55e3dc9d79c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbee49f1c2f8d13 server_cookie=6fc4897761299def in_seq=0 out_seq=0 2024-09-12T02:55:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.274+0000 7f63917fa640 10 client.25358 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63917fa640 1 -- 172.21.15.46:0/863243993 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3dc8b6d70 con 0x55e3dc9d55d0 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63917fa640 10 client.25358 handle_command_reply: tid=0 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 client.25358 shutdown 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 msgr2=0x55e3dc95e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc95e4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63a40276d0 tx=0x7f63a401f800 comp rx=0 tx=0).stop 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 shutdown_connections 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e3dc9d55d0 0x55e3dc9d79c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3dc955cb0 0x55e3dc94ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63917fa640 0 client.25358 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3dc94b410 0x55e3dc94f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 --2- 172.21.15.46:0/863243993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e3dc94aaf0 0x55e3dc95e4f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 >> 172.21.15.46:0/863243993 conn(0x55e3dc947060 msgr2=0x55e3dc94c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 shutdown_connections 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 1 -- 172.21.15.46:0/863243993 wait complete. 2024-09-12T02:55:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.281+0000 7f63b2a09740 20 client.25358 trim_cache size 0 max 16384 2024-09-12T02:55:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:55:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:55:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:55:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 msgr2=0x7f63940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 0x7f63940a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b50 tx=0x7f63a4005f90 comp rx=0 tx=0).stop 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 msgr2=0x7f63a81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6398005940 tx=0x7f63980042f0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 shutdown_connections 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63940a2da0 0x7f63940a5260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63a8107b70 0x7f63a819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a8102470 0x7f63a81987e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 --2- 172.21.15.46:0/3759666064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a8101ad0 0x7f63a81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 >> 172.21.15.46:0/3759666064 conn(0x7f63a80fd7d0 msgr2=0x7f63a80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 shutdown_connections 2024-09-12T02:55:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:15.282+0000 7f63afb91640 1 -- 172.21.15.46:0/3759666064 wait complete. 2024-09-12T02:55:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:15 smithi196 ceph-mon[34587]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:55:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:15 smithi046 ceph-mon[30951]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:55:16.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:15 smithi145 ceph-mon[34674]: pgmap v155: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:55:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:17 smithi196 ceph-mon[34587]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:55:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:17 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:17 smithi046 ceph-mon[30951]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:55:18.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:17 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:17 smithi145 ceph-mon[34674]: pgmap v156: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:55:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:17 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:18 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:18.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:18 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:18.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:18 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:55:19.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:18 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:19.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:18 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:19.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:18 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:55:19.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:18 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:19.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:18 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:19.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:18 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:55:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:19 smithi145 ceph-mon[34674]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:20.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:19 smithi196 ceph-mon[34587]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:20.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:19 smithi046 ceph-mon[30951]: pgmap v157: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:22.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:21 smithi145 ceph-mon[34674]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:22.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:21 smithi196 ceph-mon[34587]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:22.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:21 smithi046 ceph-mon[30951]: pgmap v158: 129 pgs: 129 active+clean; 5.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:24.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:23 smithi046 ceph-mon[30951]: pgmap v159: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:24.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:23 smithi145 ceph-mon[34674]: pgmap v159: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:23 smithi196 ceph-mon[34587]: pgmap v159: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:26.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:25 smithi145 ceph-mon[34674]: pgmap v160: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:26.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:25 smithi196 ceph-mon[34587]: pgmap v160: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:26.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:25 smithi046 ceph-mon[30951]: pgmap v160: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:28.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:27 smithi145 ceph-mon[34674]: pgmap v161: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:28.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:27 smithi196 ceph-mon[34587]: pgmap v161: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:28.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:27 smithi046 ceph-mon[30951]: pgmap v161: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- 172.21.15.46:0/4076865129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400101ad0 msgr2=0x7f6400101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 --2- 172.21.15.46:0/4076865129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400101ad0 0x7f6400101f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009930 tx=0x7f63f002a420 comp rx=0 tx=0).stop 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- 172.21.15.46:0/4076865129 shutdown_connections 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 --2- 172.21.15.46:0/4076865129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400102470 0x7f640010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 --2- 172.21.15.46:0/4076865129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400101ad0 0x7f6400101f30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 --2- 172.21.15.46:0/4076865129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400107b70 0x7f6400107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- 172.21.15.46:0/4076865129 >> 172.21.15.46:0/4076865129 conn(0x7f64000fd7d0 msgr2=0x7f64000ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- 172.21.15.46:0/4076865129 shutdown_connections 2024-09-12T02:55:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- 172.21.15.46:0/4076865129 wait complete. 2024-09-12T02:55:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 Processor -- start 2024-09-12T02:55:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.386+0000 7f640770a640 1 -- start start 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 0x7f64001982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102470 0x7f64001987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111d90 con 0x7f6400107b70 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111c10 con 0x7f6400101ad0 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f640770a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111f10 con 0x7f6400102470 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56292/0 (socket says 172.21.15.46:56292) 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 -- 172.21.15.46:0/593729492 learned_addr learned my addr 172.21.15.46:0/593729492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6404ca5640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102470 0x7f64001987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f64054a6640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 0x7f64001982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 -- 172.21.15.46:0/593729492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102470 msgr2=0x7f64001987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102470 0x7f64001987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 -- 172.21.15.46:0/593729492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 msgr2=0x7f64001982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 0x7f64001982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 -- 172.21.15.46:0/593729492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640019d290 con 0x7f6400107b70 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f64054a6640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 0x7f64001982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.387+0000 7f6405ca7640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f63f800eaa0 tx=0x7f63f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.388+0000 7f63f67fc640 1 -- 172.21.15.46:0/593729492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800cdb0 con 0x7f6400107b70 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.388+0000 7f63f67fc640 1 -- 172.21.15.46:0/593729492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63f8005020 con 0x7f6400107b70 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.388+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640019d580 con 0x7f6400107b70 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.388+0000 7f63f67fc640 1 -- 172.21.15.46:0/593729492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f80116d0 con 0x7f6400107b70 2024-09-12T02:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.388+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a4e10 con 0x7f6400107b70 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f63f67fc640 1 -- 172.21.15.46:0/593729492 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63f80051d0 con 0x7f6400107b70 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 Processor -- start 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 -- start start 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33364aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c3336580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3335af2f0 con 0x55c33364b410 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c333557270 con 0x55c333655cb0 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f640a49e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3335ab2b0 con 0x55c33364aaf0 2024-09-12T02:55:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56294/0 (socket says 172.21.15.46:56294) 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 -- 172.21.15.46:0/3887106983 learned_addr learned my addr 172.21.15.46:0/3887106983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33364aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c3336580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f63f67fc640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 0x7f63ec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 -- 172.21.15.46:0/3887106983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 msgr2=0x55c33364aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33364aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 -- 172.21.15.46:0/3887106983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 msgr2=0x55c3336580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c3336580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.391+0000 7f64054a6640 1 -- 172.21.15.46:0/3887106983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c333556340 con 0x55c33364b410 2024-09-12T02:55:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c3336580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f6405ca7640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 0x7f63ec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f63f67fc640 1 -- 172.21.15.46:0/593729492 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63f80cf900 con 0x7f6400107b70 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f64054a6640 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00dc40 tx=0x7f63fc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2225f5454ba6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f63eb7fe640 1 -- 172.21.15.46:0/3887106983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc005020 con 0x55c33364b410 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f63eb7fe640 1 -- 172.21.15.46:0/3887106983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63fc0047a0 con 0x55c33364b410 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.392+0000 7f63eb7fe640 1 -- 172.21.15.46:0/3887106983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0116a0 con 0x55c33364b410 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- 172.21.15.46:0/3887106983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 msgr2=0x55c333655770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f6405ca7640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 0x7f63ec0a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63f80a4200 tx=0x7f63f8086040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00dc40 tx=0x7f63fc00f730 comp rx=0 tx=0).stop 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- 172.21.15.46:0/3887106983 shutdown_connections 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c3336580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333655770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 --2- 172.21.15.46:0/3887106983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33364aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- 172.21.15.46:0/3887106983 >> 172.21.15.46:0/3887106983 conn(0x55c333647060 msgr2=0x55c333647440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- 172.21.15.46:0/3887106983 shutdown_connections 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- 172.21.15.46:0/3887106983 wait complete. 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 -- start start 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.393+0000 7f640a49e740 1 Processor -- start 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c333652bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333653670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c333658880 con 0x55c33364b410 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c333538290 con 0x55c333655cb0 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c333658a00 con 0x55c33364aaf0 2024-09-12T02:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f640a49e740 10 client.0 _ll_register_callbacks cb 0x55c333569140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56298/0 (socket says 172.21.15.46:56298) 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 -- 172.21.15.46:0/2210754068 learned_addr learned my addr 172.21.15.46:0/2210754068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f6404ca5640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c333652bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f6404ca5640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333653670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 -- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 msgr2=0x55c333652bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c333652bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 -- 172.21.15.46:0/2210754068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 msgr2=0x55c333653670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333653670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f64054a6640 1 -- 172.21.15.46:0/2210754068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3332c2210 con 0x55c33364b410 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.394+0000 7f6404ca5640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c333652bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f64054a6640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a1e3232c083340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc02a060 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63fc004b60 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc0047a0 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63fc031050 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c33338e620 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63fc0047a0 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63fc0035f0 con 0x55c33364b410 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.395+0000 7f640a49e740 20 client.15759 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c333665c00 0x55c333667ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f640a49e740 4 client.15759 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3335bca20 con 0x55c333665c00 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f6404ca5640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c333665c00 0x55c333667ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f63e8ff9640 10 client.15759 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.396+0000 7f6404ca5640 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c333665c00 0x55c333667ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4e4c162b94f26c server_cookie=a4a0496b1be9affc in_seq=0 out_seq=0 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f63e8ff9640 1 -- 172.21.15.46:0/2210754068 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3335bca20 con 0x55c333665c00 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f63e8ff9640 10 client.15759 handle_command_reply: tid=0 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 client.15759 shutdown 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 msgr2=0x55c333653130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc00d9c0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 shutdown_connections 2024-09-12T02:55:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c333665c00 0x55c333667ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333653670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333653130 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f63e8ff9640 0 client.15759 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 --2- 172.21.15.46:0/2210754068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c333652bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 >> 172.21.15.46:0/2210754068 conn(0x55c333647060 msgr2=0x55c3336461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 shutdown_connections 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 1 -- 172.21.15.46:0/2210754068 wait complete. 2024-09-12T02:55:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.397+0000 7f640a49e740 20 client.15759 trim_cache size 0 max 16384 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.414+0000 7f640a49e740 1 Processor -- start 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.414+0000 7f640a49e740 1 -- start start 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.414+0000 7f640a49e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 0x55c3336534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f640a49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f640a49e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c333655cb0 0x55c33366db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f640a49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b16b0 con 0x55c33364b410 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f640a49e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64001aef80 con 0x55c33364aaf0 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f640a49e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64001acdd0 con 0x55c333655cb0 2024-09-12T02:55:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56312/0 (socket says 172.21.15.46:56312) 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 -- 172.21.15.46:0/3244934903 learned_addr learned my addr 172.21.15.46:0/3244934903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 0x55c3336534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c333655cb0 0x55c33366db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 -- 172.21.15.46:0/3244934903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c333655cb0 msgr2=0x55c33366db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c333655cb0 0x55c33366db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 -- 172.21.15.46:0/3244934903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 msgr2=0x55c3336534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 0x55c3336534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 -- 172.21.15.46:0/3244934903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33364b240 con 0x55c33364b410 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f6404ca5640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 0x55c3336534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f64054a6640 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f63fc003b60 tx=0x7f63fc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f87f58db00b568 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f63eaffd640 1 -- 172.21.15.46:0/3244934903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc01f070 con 0x55c33364b410 2024-09-12T02:55:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f63eaffd640 1 -- 172.21.15.46:0/3244934903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63fc00fc30 con 0x55c33364b410 2024-09-12T02:55:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.415+0000 7f63eaffd640 1 -- 172.21.15.46:0/3244934903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004000 con 0x55c33364b410 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 -- 172.21.15.46:0/3244934903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 msgr2=0x55c33366d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f63fc003b60 tx=0x7f63fc003b90 comp rx=0 tx=0).stop 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 -- 172.21.15.46:0/3244934903 shutdown_connections 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c333655cb0 0x55c33366db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c33366d5c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 --2- 172.21.15.46:0/3244934903 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c33364aaf0 0x55c3336534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 -- 172.21.15.46:0/3244934903 >> 172.21.15.46:0/3244934903 conn(0x55c333647060 msgr2=0x55c333651cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 -- 172.21.15.46:0/3244934903 shutdown_connections 2024-09-12T02:55:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.416+0000 7f640a49e740 1 -- 172.21.15.46:0/3244934903 wait complete. 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 -- start start 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 Processor -- start 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33365e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333659190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ad440 con 0x55c33364b410 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ae8b0 con 0x55c333655cb0 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64001adb10 con 0x55c33364aaf0 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f640a49e740 10 client.0 _ll_register_callbacks cb 0x7f63fc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f64054a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56320/0 (socket says 172.21.15.46:56320) 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.417+0000 7f64054a6640 1 -- 172.21.15.46:0/1323725954 learned_addr learned my addr 172.21.15.46:0/1323725954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f6404ca5640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33365e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f6404ca5640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333659190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 -- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 msgr2=0x55c33365e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33365e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 -- 172.21.15.46:0/1323725954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 msgr2=0x55c333659190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333659190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 -- 172.21.15.46:0/1323725954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c333659780 con 0x55c33364b410 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f6404ca5640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333659190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f64054a6640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871531d0e4a0af85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f63e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc01f040 con 0x55c33364b410 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f63fc005140 con 0x55c33364b410 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc00fc30 con 0x55c33364b410 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63fc049050 con 0x55c33364b410 2024-09-12T02:55:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f640a49e740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.418+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c3336cf390 con 0x55c33364b410 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63fc00fc30 con 0x55c33364b410 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63fc026450 con 0x55c33364b410 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 20 client.15765 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c3336cfa80 0x55c3336d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 4 client.15765 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3335b6d70 con 0x55c3336cfa80 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f6404ca5640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c3336cfa80 0x55c3336d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f63e8ff9640 10 client.15765 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.419+0000 7f6404ca5640 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c3336cfa80 0x55c3336d1ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29370b2314cbff server_cookie=8fd7bbbd79709c5e in_seq=0 out_seq=0 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f63e8ff9640 1 -- 172.21.15.46:0/1323725954 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3335b6d70 con 0x55c3336cfa80 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f63e8ff9640 10 client.15765 handle_command_reply: tid=0 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 client.15765 shutdown 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 msgr2=0x55c333658c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc003e60 comp rx=0 tx=0).stop 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 shutdown_connections 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c3336cfa80 0x55c3336d1ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c333655cb0 0x55c333659190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c33364b410 0x55c333658c50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f63e8ff9640 0 client.15765 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 --2- 172.21.15.46:0/1323725954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c33364aaf0 0x55c33365e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 >> 172.21.15.46:0/1323725954 conn(0x55c333647060 msgr2=0x55c33364c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 shutdown_connections 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 1 -- 172.21.15.46:0/1323725954 wait complete. 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.427+0000 7f640a49e740 20 client.15765 trim_cache size 0 max 16384 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:55:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:55:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 msgr2=0x7f63ec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 0x7f63ec0a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63f80a4200 tx=0x7f63f8086040 comp rx=0 tx=0).stop 2024-09-12T02:55:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 msgr2=0x7f640019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f63f800eaa0 tx=0x7f63f800ef60 comp rx=0 tx=0).stop 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 shutdown_connections 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f63ec0a2e20 0x7f63ec0a52e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107b70 0x7f640019cb60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102470 0x7f64001987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 --2- 172.21.15.46:0/593729492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6400101ad0 0x7f64001982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 >> 172.21.15.46:0/593729492 conn(0x7f64000fd7d0 msgr2=0x7f64000fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 shutdown_connections 2024-09-12T02:55:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:28.428+0000 7f640770a640 1 -- 172.21.15.46:0/593729492 wait complete. 2024-09-12T02:55:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:29 smithi145 ceph-mon[34674]: pgmap v162: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:30.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:29 smithi196 ceph-mon[34587]: pgmap v162: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:30.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:29 smithi046 ceph-mon[30951]: pgmap v162: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:32.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:31 smithi145 ceph-mon[34674]: pgmap v163: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:32.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:31 smithi196 ceph-mon[34587]: pgmap v163: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:32.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:31 smithi046 ceph-mon[30951]: pgmap v163: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:33 smithi145 ceph-mon[34674]: pgmap v164: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:33 smithi196 ceph-mon[34587]: pgmap v164: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:34.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:33 smithi046 ceph-mon[30951]: pgmap v164: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:36.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:35 smithi145 ceph-mon[34674]: pgmap v165: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-12T02:55:36.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:35 smithi196 ceph-mon[34587]: pgmap v165: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-12T02:55:36.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:35 smithi046 ceph-mon[30951]: pgmap v165: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 16 op/s 2024-09-12T02:55:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:37 smithi145 ceph-mon[34674]: pgmap v166: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:37 smithi196 ceph-mon[34587]: pgmap v166: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:38.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:37 smithi046 ceph-mon[30951]: pgmap v166: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-12T02:55:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:39 smithi145 ceph-mon[34674]: pgmap v167: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:39 smithi196 ceph-mon[34587]: pgmap v167: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:40.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:39 smithi046 ceph-mon[30951]: pgmap v167: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 -- 172.21.15.46:0/1211744831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c103f60 msgr2=0x7fc16c104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 --2- 172.21.15.46:0/1211744831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c103f60 0x7fc16c104340 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc16802a410 comp rx=0 tx=0).stop 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 -- 172.21.15.46:0/1211744831 shutdown_connections 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 --2- 172.21.15.46:0/1211744831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c1090b0 0x7fc16c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 --2- 172.21.15.46:0/1211744831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c104910 0x7fc16c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 --2- 172.21.15.46:0/1211744831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c103f60 0x7fc16c104340 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 -- 172.21.15.46:0/1211744831 >> 172.21.15.46:0/1211744831 conn(0x7fc16c0ff970 msgr2=0x7fc16c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 -- 172.21.15.46:0/1211744831 shutdown_connections 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.520+0000 7fc17405e640 1 -- 172.21.15.46:0/1211744831 wait complete. 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 Processor -- start 2024-09-12T02:55:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 -- start start 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c103f60 0x7fc16c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 0x7fc16c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c113fb0 con 0x7fc16c1090b0 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c113e30 con 0x7fc16c103f60 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc17405e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c114130 con 0x7fc16c19ed80 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc1715f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc1715f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57872/0 (socket says 172.21.15.46:57872) 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc1715f9640 1 -- 172.21.15.46:0/115612343 learned_addr learned my addr 172.21.15.46:0/115612343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.521+0000 7fc171dfa640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c103f60 0x7fc16c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1725fb640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 0x7fc16c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 -- 172.21.15.46:0/115612343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 msgr2=0x7fc16c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 0x7fc16c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 -- 172.21.15.46:0/115612343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c103f60 msgr2=0x7fc16c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c103f60 0x7fc16c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 -- 172.21.15.46:0/115612343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c19f870 con 0x7fc16c1090b0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1725fb640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 0x7fc16c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc1715f9640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00d770 tx=0x7fc15c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc162ffd640 1 -- 172.21.15.46:0/115612343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c017070 con 0x7fc16c1090b0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc162ffd640 1 -- 172.21.15.46:0/115612343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc15c009c30 con 0x7fc16c1090b0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c19fb60 con 0x7fc16c1090b0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc162ffd640 1 -- 172.21.15.46:0/115612343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00f620 con 0x7fc16c1090b0 2024-09-12T02:55:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.522+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1a73a0 con 0x7fc16c1090b0 2024-09-12T02:55:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.525+0000 7fc162ffd640 1 -- 172.21.15.46:0/115612343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc15c01a070 con 0x7fc16c1090b0 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.525+0000 7fc176df8740 1 Processor -- start 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.525+0000 7fc176df8740 1 -- start start 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cbced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 0x564b87cca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b87c212f0 con 0x564b87cbd410 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564b87bc9270 con 0x564b87cc7cb0 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc176df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b87c1d2b0 con 0x564b87cbcaf0 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc162ffd640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 0x7fc1580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57878/0 (socket says 172.21.15.46:57878) 2024-09-12T02:55:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 -- 172.21.15.46:0/218007598 learned_addr learned my addr 172.21.15.46:0/218007598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc1715f9640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 0x564b87cca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc1725fb640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cbced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc1725fb640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 0x7fc1580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc162ffd640 1 -- 172.21.15.46:0/115612343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc15c0cf240 con 0x7fc16c1090b0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 -- 172.21.15.46:0/218007598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 msgr2=0x564b87cbced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cbced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 -- 172.21.15.46:0/218007598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 msgr2=0x564b87cca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 0x564b87cca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc171dfa640 1 -- 172.21.15.46:0/218007598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b87bc8340 con 0x564b87cbd410 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.526+0000 7fc1715f9640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 0x564b87cca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.527+0000 7fc171dfa640 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc168032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9389cdce2cba0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.527+0000 7fc1725fb640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 0x7fc1580a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc164005c40 tx=0x7fc164009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.527+0000 7fc157fff640 1 -- 172.21.15.46:0/218007598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168040070 con 0x564b87cbd410 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.527+0000 7fc157fff640 1 -- 172.21.15.46:0/218007598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc168004030 con 0x564b87cbd410 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.527+0000 7fc157fff640 1 -- 172.21.15.46:0/218007598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168035040 con 0x564b87cbd410 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- 172.21.15.46:0/218007598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 msgr2=0x564b87cc7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc168009900 tx=0x7fc168032bb0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- 172.21.15.46:0/218007598 shutdown_connections 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cc7cb0 0x564b87cca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cc7770 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 --2- 172.21.15.46:0/218007598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cbced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- 172.21.15.46:0/218007598 >> 172.21.15.46:0/218007598 conn(0x564b87cb9060 msgr2=0x564b87cb9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- 172.21.15.46:0/218007598 shutdown_connections 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- 172.21.15.46:0/218007598 wait complete. 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 -- start start 2024-09-12T02:55:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.528+0000 7fc176df8740 1 Processor -- start 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cc5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b87cca880 con 0x564b87cbcaf0 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564b87baa290 con 0x564b87cbd410 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b87ccaa00 con 0x564b87cc7cb0 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc176df8740 10 client.0 _ll_register_callbacks cb 0x564b87bdb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57880/0 (socket says 172.21.15.46:57880) 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 -- 172.21.15.46:0/123451463 learned_addr learned my addr 172.21.15.46:0/123451463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1725fb640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1715f9640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 -- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 msgr2=0x564b87cc55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 -- 172.21.15.46:0/123451463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 msgr2=0x564b87cc5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cc5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 -- 172.21.15.46:0/123451463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b87934210 con 0x564b87cbcaf0 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1725fb640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1715f9640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cc5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc171dfa640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc168005390 tx=0x7fc168031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ef5379590a31c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.529+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168040070 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc168004030 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc1680314a0 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc168056050 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b87a00620 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc168004030 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1680035f0 con 0x564b87cbcaf0 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 20 client.15777 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87cd7a60 0x564b87cd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 4 client.15777 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.530+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b87c2ea20 con 0x564b87cd7a60 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.531+0000 7fc1715f9640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87cd7a60 0x564b87cd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.531+0000 7fc1715f9640 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87cd7a60 0x564b87cd9e50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77069fc0cbe66ca server_cookie=a3ab0811d493fea1 in_seq=0 out_seq=0 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.531+0000 7fc1557fa640 10 client.15777 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc1557fa640 1 -- 172.21.15.46:0/123451463 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b87c2ea20 con 0x564b87cd7a60 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc1557fa640 10 client.15777 handle_command_reply: tid=0 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 client.15777 shutdown 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 msgr2=0x564b87cc4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc168005390 tx=0x7fc168031400 comp rx=0 tx=0).stop 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 shutdown_connections 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87cd7a60 0x564b87cd9e50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cc5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 --2- 172.21.15.46:0/123451463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbcaf0 0x564b87cc4b40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc1557fa640 0 client.15777 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 >> 172.21.15.46:0/123451463 conn(0x564b87cb9060 msgr2=0x564b87cbed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 shutdown_connections 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 1 -- 172.21.15.46:0/123451463 wait complete. 2024-09-12T02:55:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.532+0000 7fc176df8740 20 client.15777 trim_cache size 0 max 16384 2024-09-12T02:55:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 Processor -- start 2024-09-12T02:55:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 -- start start 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cc6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cbf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b3790 con 0x564b87cc7cb0 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b1060 con 0x564b87cbd410 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc176df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1aeeb0 con 0x564b87cbcaf0 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc1725fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc1725fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57892/0 (socket says 172.21.15.46:57892) 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.554+0000 7fc1725fb640 1 -- 172.21.15.46:0/4073966993 learned_addr learned my addr 172.21.15.46:0/4073966993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1715f9640 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cbf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc171dfa640 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cc6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 -- 172.21.15.46:0/4073966993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 msgr2=0x564b87cc6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cc6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 -- 172.21.15.46:0/4073966993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 msgr2=0x564b87cbf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cbf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 -- 172.21.15.46:0/4073966993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b87cb7790 con 0x564b87cc7cb0 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1725fb640 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e8e0 tx=0x7fc1640099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842e5f77f04453d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1577fe640 1 -- 172.21.15.46:0/4073966993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164018070 con 0x564b87cc7cb0 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1577fe640 1 -- 172.21.15.46:0/4073966993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16400ee60 con 0x564b87cc7cb0 2024-09-12T02:55:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.555+0000 7fc1577fe640 1 -- 172.21.15.46:0/4073966993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400f660 con 0x564b87cc7cb0 2024-09-12T02:55:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- 172.21.15.46:0/4073966993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 msgr2=0x564b87cc53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e8e0 tx=0x7fc1640099e0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- 172.21.15.46:0/4073966993 shutdown_connections 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cc7cb0 0x564b87cc53f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbd410 0x564b87cbf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 --2- 172.21.15.46:0/4073966993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cbcaf0 0x564b87cc6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- 172.21.15.46:0/4073966993 >> 172.21.15.46:0/4073966993 conn(0x564b87cb9060 msgr2=0x564b87cbbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- 172.21.15.46:0/4073966993 shutdown_connections 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- 172.21.15.46:0/4073966993 wait complete. 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.556+0000 7fc176df8740 1 -- start start 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 Processor -- start 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 0x564b87ccff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1af520 con 0x564b87cbd410 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b0990 con 0x564b87cbcaf0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1afbf0 con 0x564b87cc7cb0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc176df8740 10 client.0 _ll_register_callbacks cb 0x7fc168032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57908/0 (socket says 172.21.15.46:57908) 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 -- 172.21.15.46:0/1769663418 learned_addr learned my addr 172.21.15.46:0/1769663418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc171dfa640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 0x564b87ccff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1725fb640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 -- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 msgr2=0x564b87cc1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 -- 172.21.15.46:0/1769663418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 msgr2=0x564b87ccff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 0x564b87ccff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc1715f9640 1 -- 172.21.15.46:0/1769663418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b87cc1cb0 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.557+0000 7fc171dfa640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 0x564b87ccff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1715f9640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ccc30 tx=0x7fc15c06d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee358bbfc2012a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c021040 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc15c097a70 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16400ee60 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc15c09b400 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b87d42020 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16400ee60 con 0x564b87cbd410 2024-09-12T02:55:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc15c097150 con 0x564b87cbd410 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 20 client.15783 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87d427a0 0x564b87d44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 4 client.15783 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.558+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b87c28d70 con 0x564b87d427a0 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.559+0000 7fc171dfa640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87d427a0 0x564b87d44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.559+0000 7fc1557fa640 10 client.15783 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.559+0000 7fc171dfa640 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87d427a0 0x564b87d44bc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c87854178721d8 server_cookie=3871edd21da1f1e6 in_seq=0 out_seq=0 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.566+0000 7fc1557fa640 1 -- 172.21.15.46:0/1769663418 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b87c28d70 con 0x564b87d427a0 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.566+0000 7fc1557fa640 10 client.15783 handle_command_reply: tid=0 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.566+0000 7fc176df8740 1 client.15783 shutdown 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.566+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 msgr2=0x564b87cd04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.566+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ccc30 tx=0x7fc15c06d3c0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 shutdown_connections 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564b87d427a0 0x564b87d44bc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b87cc7cb0 0x564b87cc1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b87cbd410 0x564b87cd04d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 --2- 172.21.15.46:0/1769663418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b87cbcaf0 0x564b87ccff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc1557fa640 0 client.15783 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 >> 172.21.15.46:0/1769663418 conn(0x564b87cb9060 msgr2=0x564b87cb79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 shutdown_connections 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 1 -- 172.21.15.46:0/1769663418 wait complete. 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc176df8740 20 client.15783 trim_cache size 0 max 16384 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:55:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:55:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:55:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:55:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 msgr2=0x7fc1580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 0x7fc1580a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc164005c40 tx=0x7fc164009500 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 msgr2=0x7fc16c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00d770 tx=0x7fc15c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 shutdown_connections 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.567+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc1580a2e20 0x7fc1580a52e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c19ed80 0x7fc16c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc16c1090b0 0x7fc16c19aa00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 --2- 172.21.15.46:0/115612343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c103f60 0x7fc16c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 >> 172.21.15.46:0/115612343 conn(0x7fc16c0ff970 msgr2=0x7fc16c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 shutdown_connections 2024-09-12T02:55:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:41.568+0000 7fc17405e640 1 -- 172.21.15.46:0/115612343 wait complete. 2024-09-12T02:55:42.084 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:41 smithi046 ceph-mon[30951]: pgmap v168: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:41 smithi145 ceph-mon[34674]: pgmap v168: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:42.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:41 smithi196 ceph-mon[34587]: pgmap v168: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:43 smithi145 ceph-mon[34674]: pgmap v169: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:44.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:43 smithi196 ceph-mon[34587]: pgmap v169: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:44.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:43 smithi046 ceph-mon[30951]: pgmap v169: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:45 smithi145 ceph-mon[34674]: pgmap v170: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:46.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:45 smithi196 ceph-mon[34587]: pgmap v170: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:46.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:45 smithi046 ceph-mon[30951]: pgmap v170: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:55:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:47 smithi145 ceph-mon[34674]: pgmap v171: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:47 smithi196 ceph-mon[34587]: pgmap v171: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:48.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:47 smithi046 ceph-mon[30951]: pgmap v171: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:50.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:49 smithi145 ceph-mon[34674]: pgmap v172: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:49 smithi196 ceph-mon[34587]: pgmap v172: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:50.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:49 smithi046 ceph-mon[30951]: pgmap v172: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:52.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:52 smithi196 ceph-mon[34587]: pgmap v173: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:52.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:52 smithi046 ceph-mon[30951]: pgmap v173: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:52 smithi145 ceph-mon[34674]: pgmap v173: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:54.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:54 smithi196 ceph-mon[34587]: pgmap v174: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:54 smithi046 ceph-mon[30951]: pgmap v174: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:54 smithi145 ceph-mon[34674]: pgmap v174: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:55:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 -- 172.21.15.46:0/4285189504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc107b70 msgr2=0x7ff5cc107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 --2- 172.21.15.46:0/4285189504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc107b70 0x7ff5cc107f50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009930 tx=0x7ff5c802a420 comp rx=0 tx=0).stop 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 -- 172.21.15.46:0/4285189504 shutdown_connections 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 --2- 172.21.15.46:0/4285189504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc102470 0x7ff5cc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 --2- 172.21.15.46:0/4285189504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 --2- 172.21.15.46:0/4285189504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc107b70 0x7ff5cc107f50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.667+0000 7ff5d3265640 1 -- 172.21.15.46:0/4285189504 >> 172.21.15.46:0/4285189504 conn(0x7ff5cc0fd7d0 msgr2=0x7ff5cc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.668+0000 7ff5d3265640 1 -- 172.21.15.46:0/4285189504 shutdown_connections 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.668+0000 7ff5d3265640 1 -- 172.21.15.46:0/4285189504 wait complete. 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.668+0000 7ff5d3265640 1 Processor -- start 2024-09-12T02:55:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.668+0000 7ff5d3265640 1 -- start start 2024-09-12T02:55:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.668+0000 7ff5d3265640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 0x7ff5cc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc107b70 0x7ff5cc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111d90 con 0x7ff5cc101ad0 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111c10 con 0x7ff5cc107b70 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111f10 con 0x7ff5cc102470 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55442/0 (socket says 172.21.15.46:55442) 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 -- 172.21.15.46:0/664491291 learned_addr learned my addr 172.21.15.46:0/664491291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1802640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc107b70 0x7ff5cc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d0800640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 0x7ff5cc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 -- 172.21.15.46:0/664491291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 msgr2=0x7ff5cc198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 0x7ff5cc198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 -- 172.21.15.46:0/664491291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc107b70 msgr2=0x7ff5cc19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc107b70 0x7ff5cc19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 -- 172.21.15.46:0/664491291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc19d2d0 con 0x7ff5cc101ad0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d0800640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 0x7ff5cc198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d1001640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005940 tx=0x7ff5c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5c27fc640 1 -- 172.21.15.46:0/664491291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c803f030 con 0x7ff5cc101ad0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5c27fc640 1 -- 172.21.15.46:0/664491291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c80318a0 con 0x7ff5cc101ad0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.669+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc19d560 con 0x7ff5cc101ad0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.670+0000 7ff5c27fc640 1 -- 172.21.15.46:0/664491291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8035070 con 0x7ff5cc101ad0 2024-09-12T02:55:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.670+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a4e50 con 0x7ff5cc101ad0 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.672+0000 7ff5c27fc640 1 -- 172.21.15.46:0/664491291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff5c8042030 con 0x7ff5cc101ad0 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 Processor -- start 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 -- start start 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5c27fc640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 0x7ff5b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d1802640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 0x7ff5b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 0x55c2b80300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c2b7f872f0 con 0x55c2b8022af0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c2b7f2f270 con 0x55c2b8023410 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5d6004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2b7f832b0 con 0x55c2b802dcb0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.673+0000 7ff5c27fc640 1 -- 172.21.15.46:0/664491291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff5c80efec0 con 0x7ff5cc101ad0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:53674/0 (socket says 172.21.15.46:53674) 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d0800640 1 -- 172.21.15.46:0/4089674672 learned_addr learned my addr 172.21.15.46:0/4089674672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55448/0 (socket says 172.21.15.46:55448) 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1802640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 0x55c2b80300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 -- 172.21.15.46:0/4089674672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 msgr2=0x55c2b80300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 0x55c2b80300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 -- 172.21.15.46:0/4089674672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 msgr2=0x55c2b802d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 -- 172.21.15.46:0/4089674672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b7f2e340 con 0x55c2b8022af0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1802640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 0x55c2b80300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d0800640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1802640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 0x7ff5b80a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c4009a60 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5d1001640 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80ed2b0 tx=0x7ff5c80bbf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249879649a3375b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5b77fe640 1 -- 172.21.15.46:0/4089674672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8049040 con 0x55c2b8022af0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5b77fe640 1 -- 172.21.15.46:0/4089674672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c80c0430 con 0x55c2b8022af0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.674+0000 7ff5b77fe640 1 -- 172.21.15.46:0/4089674672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b86f0 con 0x55c2b8022af0 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 -- 172.21.15.46:0/4089674672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 msgr2=0x55c2b8022ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80ed2b0 tx=0x7ff5c80bbf80 comp rx=0 tx=0).stop 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 -- 172.21.15.46:0/4089674672 shutdown_connections 2024-09-12T02:55:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b802dcb0 0x55c2b80300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8023410 0x55c2b802d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 --2- 172.21.15.46:0/4089674672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8022af0 0x55c2b8022ed0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 -- 172.21.15.46:0/4089674672 >> 172.21.15.46:0/4089674672 conn(0x55c2b801f060 msgr2=0x55c2b801f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 -- 172.21.15.46:0/4089674672 shutdown_connections 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.675+0000 7ff5d6004740 1 -- 172.21.15.46:0/4089674672 wait complete. 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 -- start start 2024-09-12T02:55:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 Processor -- start 2024-09-12T02:55:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b802aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c2b8030640 con 0x55c2b802dcb0 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c2b7f10290 con 0x55c2b8022af0 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2b80307c0 con 0x55c2b8023410 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d6004740 10 client.0 _ll_register_callbacks cb 0x55c2b7f41140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d1802640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d1802640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55464/0 (socket says 172.21.15.46:55464) 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d1802640 1 -- 172.21.15.46:0/1985242821 learned_addr learned my addr 172.21.15.46:0/1985242821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.676+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b802aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 -- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 msgr2=0x55c2b802b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 -- 172.21.15.46:0/1985242821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 msgr2=0x55c2b802b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 -- 172.21.15.46:0/1985242821 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b7c9a210 con 0x55c2b8022af0 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1802640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b802aba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8031440 tx=0x7ff5c80bc140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caef4b919cdaad35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:55:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80d4020 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c80c0430 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff5c80b86f0 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5c8049070 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d6004740 10 client.25403 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.677+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2b7d66620 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c80c0430 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5c80c4960 con 0x55c2b8022af0 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 10 client.25403 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 20 client.25403 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b803d7e0 0x55c2b803fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 4 client.25403 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2b7f94a20 con 0x55c2b803d7e0 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.678+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b803d7e0 0x55c2b803fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b803d7e0 0x55c2b803fbd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d5fb84a3908005 server_cookie=ca520d00f862a54f in_seq=0 out_seq=0 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5b4ff9640 10 client.25403 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1985242821 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2b7f94a20 con 0x55c2b803d7e0 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5b4ff9640 10 client.25403 handle_command_reply: tid=0 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 client.25403 shutdown 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 msgr2=0x55c2b802aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b802aba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8031440 tx=0x7ff5c80bc140 comp rx=0 tx=0).stop 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 shutdown_connections 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b803d7e0 0x55c2b803fbd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b802b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1985242821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b802aba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5b4ff9640 0 client.25403 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.679+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 >> 172.21.15.46:0/1985242821 conn(0x55c2b801f060 msgr2=0x55c2b8024dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.680+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 shutdown_connections 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.680+0000 7ff5d6004740 1 -- 172.21.15.46:0/1985242821 wait complete. 2024-09-12T02:55:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.680+0000 7ff5d6004740 20 client.25403 trim_cache size 0 max 16384 2024-09-12T02:55:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 Processor -- start 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 -- start start 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b8025560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b1690 con 0x55c2b802dcb0 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1aef60 con 0x55c2b8022af0 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d6004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1acdb0 con 0x55c2b8023410 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55470/0 (socket says 172.21.15.46:55470) 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 -- 172.21.15.46:0/231594135 learned_addr learned my addr 172.21.15.46:0/231594135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d0800640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1001640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b8025560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 -- 172.21.15.46:0/231594135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 msgr2=0x55c2b802b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 -- 172.21.15.46:0/231594135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 msgr2=0x55c2b8025560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b8025560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1802640 1 -- 172.21.15.46:0/231594135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2b8022920 con 0x55c2b802dcb0 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d1001640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b8025560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.695+0000 7ff5d0800640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.696+0000 7ff5d1802640 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e7e0 tx=0x7ff5c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8772d432ea01ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.696+0000 7ff5b6ffd640 1 -- 172.21.15.46:0/231594135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c402b070 con 0x55c2b802dcb0 2024-09-12T02:55:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.696+0000 7ff5b6ffd640 1 -- 172.21.15.46:0/231594135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c401de50 con 0x55c2b802dcb0 2024-09-12T02:55:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.696+0000 7ff5b6ffd640 1 -- 172.21.15.46:0/231594135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4026cf0 con 0x55c2b802dcb0 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- 172.21.15.46:0/231594135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 msgr2=0x55c2b8045470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e7e0 tx=0x7ff5c4015000 comp rx=0 tx=0).stop 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- 172.21.15.46:0/231594135 shutdown_connections 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b802dcb0 0x55c2b8045470 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8023410 0x55c2b802b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 --2- 172.21.15.46:0/231594135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b8022af0 0x55c2b8025560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- 172.21.15.46:0/231594135 >> 172.21.15.46:0/231594135 conn(0x55c2b801f060 msgr2=0x55c2b802cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- 172.21.15.46:0/231594135 shutdown_connections 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- 172.21.15.46:0/231594135 wait complete. 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 -- start start 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.697+0000 7ff5d6004740 1 Processor -- start 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8022af0 0x55c2b8035d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b802dcb0 0x55c2b8027620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1ad420 con 0x55c2b8023410 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1ae890 con 0x55c2b802dcb0 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1adaf0 con 0x55c2b8022af0 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d6004740 10 client.0 _ll_register_callbacks cb 0x7ff5bc009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55480/0 (socket says 172.21.15.46:55480) 2024-09-12T02:55:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 -- 172.21.15.46:0/1805979607 learned_addr learned my addr 172.21.15.46:0/1805979607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d1802640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b802dcb0 0x55c2b8027620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8022af0 0x55c2b8035d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 -- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8022af0 msgr2=0x55c2b8035d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8022af0 0x55c2b8035d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 -- 172.21.15.46:0/1805979607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b802dcb0 msgr2=0x55c2b8027620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b802dcb0 0x55c2b8027620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 -- 172.21.15.46:0/1805979607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2b801a2d0 con 0x55c2b8023410 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5d0800640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55c2b802c1e0 tx=0x7ff5bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43533b2d0d586daa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.698+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc01e620 con 0x55c2b8023410 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c401de50 con 0x55c2b8023410 2024-09-12T02:55:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff5bc01e7c0 con 0x55c2b8023410 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc02f050 con 0x55c2b8023410 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2b80a7ef0 con 0x55c2b8023410 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c401de50 con 0x55c2b8023410 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5bc012c20 con 0x55c2b8023410 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 20 client.15801 populate_metadata read hostname 'smithi046' 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b80a86a0 0x55c2b80aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 4 client.15801 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2b7f8ed70 con 0x55c2b80a86a0 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.699+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b80a86a0 0x55c2b80aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.700+0000 7ff5d1001640 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b80a86a0 0x55c2b80aaac0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa796a9988a92fee server_cookie=c876ee594e5bced7 in_seq=0 out_seq=0 2024-09-12T02:55:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.700+0000 7ff5b4ff9640 10 client.15801 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5b4ff9640 1 -- 172.21.15.46:0/1805979607 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2b7f8ed70 con 0x55c2b80a86a0 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5b4ff9640 10 client.15801 handle_command_reply: tid=0 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 client.15801 shutdown 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 msgr2=0x55c2b8036280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55c2b802c1e0 tx=0x7ff5bc0044c0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 shutdown_connections 2024-09-12T02:55:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55c2b80a86a0 0x55c2b80aaac0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c2b802dcb0 0x55c2b8027620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2b8023410 0x55c2b8036280 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 --2- 172.21.15.46:0/1805979607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2b8022af0 0x55c2b8035d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5b4ff9640 0 client.15801 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 >> 172.21.15.46:0/1805979607 conn(0x55c2b801f060 msgr2=0x55c2b801d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.707+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 shutdown_connections 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d6004740 1 -- 172.21.15.46:0/1805979607 wait complete. 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d6004740 20 client.15801 trim_cache size 0 max 16384 2024-09-12T02:55:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:55:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:55:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:55:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:55:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 msgr2=0x7ff5b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 0x7ff5b80a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c4009a60 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 msgr2=0x7ff5cc1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005940 tx=0x7ff5c80042f0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 shutdown_connections 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5b80a2e20 0x7ff5b80a52e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5cc107b70 0x7ff5cc19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5cc102470 0x7ff5cc198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 --2- 172.21.15.46:0/664491291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc101ad0 0x7ff5cc1982e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 >> 172.21.15.46:0/664491291 conn(0x7ff5cc0fd7d0 msgr2=0x7ff5cc0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.708+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 shutdown_connections 2024-09-12T02:55:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:55:54.709+0000 7ff5d3265640 1 -- 172.21.15.46:0/664491291 wait complete. 2024-09-12T02:55:56.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:56 smithi196 ceph-mon[34587]: pgmap v175: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:56.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:56 smithi046 ceph-mon[30951]: pgmap v175: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:56 smithi145 ceph-mon[34674]: pgmap v175: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:55:58.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:55:58 smithi196 ceph-mon[34587]: pgmap v176: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:55:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:55:58 smithi046 ceph-mon[30951]: pgmap v176: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:55:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:55:58 smithi145 ceph-mon[34674]: pgmap v176: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:56:00.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:00 smithi196 ceph-mon[34587]: pgmap v177: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:00.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:00 smithi046 ceph-mon[30951]: pgmap v177: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:00 smithi145 ceph-mon[34674]: pgmap v177: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:02 smithi196 ceph-mon[34587]: pgmap v178: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:02 smithi046 ceph-mon[30951]: pgmap v178: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:02 smithi145 ceph-mon[34674]: pgmap v178: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:04.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:04 smithi196 ceph-mon[34587]: pgmap v179: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:04 smithi046 ceph-mon[30951]: pgmap v179: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:04 smithi145 ceph-mon[34674]: pgmap v179: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:06.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:06 smithi196 ceph-mon[34587]: pgmap v180: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:56:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:06 smithi046 ceph-mon[30951]: pgmap v180: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:56:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:06 smithi145 ceph-mon[34674]: pgmap v180: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-09-12T02:56:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 -- 172.21.15.46:0/3986351887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 msgr2=0x7f1404100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 --2- 172.21.15.46:0/3986351887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404100e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009930 tx=0x7f13f802a420 comp rx=0 tx=0).stop 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 -- 172.21.15.46:0/3986351887 shutdown_connections 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 --2- 172.21.15.46:0/3986351887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1404101350 0x7f140410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 --2- 172.21.15.46:0/3986351887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404100e10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 --2- 172.21.15.46:0/3986351887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f14041069c0 0x7f1404106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 -- 172.21.15.46:0/3986351887 >> 172.21.15.46:0/3986351887 conn(0x7f1404077990 msgr2=0x7f14040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 -- 172.21.15.46:0/3986351887 shutdown_connections 2024-09-12T02:56:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.811+0000 7f140952a640 1 -- 172.21.15.46:0/3986351887 wait complete. 2024-09-12T02:56:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.812+0000 7f140952a640 1 Processor -- start 2024-09-12T02:56:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 -- start start 2024-09-12T02:56:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 0x7f14041976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 0x7f140419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1404110d00 con 0x7f14041009b0 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1404110b80 con 0x7f1404101350 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f140952a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1404110e80 con 0x7f14041069c0 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f1402ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f1402ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53096/0 (socket says 172.21.15.46:53096) 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f1402ffd640 1 -- 172.21.15.46:0/1964026508 learned_addr learned my addr 172.21.15.46:0/1964026508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f14027fc640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 0x7f14041976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.813+0000 7f14037fe640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 0x7f140419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 -- 172.21.15.46:0/1964026508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 msgr2=0x7f140419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 0x7f140419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 -- 172.21.15.46:0/1964026508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 msgr2=0x7f14041976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 0x7f14041976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 -- 172.21.15.46:0/1964026508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140419c180 con 0x7f14041009b0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f14037fe640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 0x7f140419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f14027fc640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 0x7f14041976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f1402ffd640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9f0 tx=0x7f13f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f13ebfff640 1 -- 172.21.15.46:0/1964026508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f400cd80 con 0x7f14041009b0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140419c470 con 0x7f14041009b0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f13ebfff640 1 -- 172.21.15.46:0/1964026508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f40108e0 con 0x7f14041009b0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f13ebfff640 1 -- 172.21.15.46:0/1964026508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4018920 con 0x7f14041009b0 2024-09-12T02:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.814+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14041a3d00 con 0x7f14041009b0 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.817+0000 7f140c2c3740 1 Processor -- start 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 -- start start 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640526332b0 0x56405263f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5640525972f0 con 0x564052632990 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56405253f270 con 0x56405263fec0 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f140c2c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5640525932b0 con 0x5640526332b0 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f13ebfff640 1 -- 172.21.15.46:0/1964026508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13f4010a90 con 0x7f14041009b0 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f14037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f14037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53110/0 (socket says 172.21.15.46:53110) 2024-09-12T02:56:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f14037fe640 1 -- 172.21.15.46:0/2418146491 learned_addr learned my addr 172.21.15.46:0/2418146491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f14027fc640 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.818+0000 7f1402ffd640 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640526332b0 0x56405263f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 -- 172.21.15.46:0/2418146491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640526332b0 msgr2=0x56405263f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640526332b0 0x56405263f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 -- 172.21.15.46:0/2418146491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 msgr2=0x5640526422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 -- 172.21.15.46:0/2418146491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405253e340 con 0x564052632990 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f14037fe640 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fb0 tx=0x7f13fc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00a82d403ffb32b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.819+0000 7f13ebfff640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 0x7f13ec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.820+0000 7f13ebfff640 1 -- 172.21.15.46:0/1964026508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f13f40cfaf0 con 0x7f14041009b0 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.820+0000 7f1402ffd640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 0x7f13ec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.820+0000 7f13e97fa640 1 -- 172.21.15.46:0/2418146491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc015ab0 con 0x564052632990 2024-09-12T02:56:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.820+0000 7f13e97fa640 1 -- 172.21.15.46:0/2418146491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13fc015c50 con 0x564052632990 2024-09-12T02:56:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.820+0000 7f13e97fa640 1 -- 172.21.15.46:0/2418146491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc01c5e0 con 0x564052632990 2024-09-12T02:56:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 -- 172.21.15.46:0/2418146491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 msgr2=0x564052632d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f1402ffd640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 0x7f13ec0a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ccf90 tx=0x7f13f40d3640 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fb0 tx=0x7f13fc010800 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 -- 172.21.15.46:0/2418146491 shutdown_connections 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640526332b0 0x56405263f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2418146491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052632d70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 -- 172.21.15.46:0/2418146491 >> 172.21.15.46:0/2418146491 conn(0x56405262f060 msgr2=0x56405262f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 -- 172.21.15.46:0/2418146491 shutdown_connections 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.821+0000 7f140c2c3740 1 -- 172.21.15.46:0/2418146491 wait complete. 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 -- start start 2024-09-12T02:56:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 Processor -- start 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564052632990 0x56405263b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5640526359a0 con 0x5640526332b0 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564052520290 con 0x56405263fec0 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564052635b20 con 0x564052632990 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f140c2c3740 10 client.0 _ll_register_callbacks cb 0x564052551140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f14027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f14027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53118/0 (socket says 172.21.15.46:53118) 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.822+0000 7f14027fc640 1 -- 172.21.15.46:0/2009257371 learned_addr learned my addr 172.21.15.46:0/2009257371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14037fe640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 -- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564052632990 msgr2=0x56405263b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564052632990 0x56405263b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 -- 172.21.15.46:0/2009257371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 msgr2=0x5640526498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 -- 172.21.15.46:0/2009257371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640522aa210 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f14027fc640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f13f8031440 tx=0x7f13f80052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508cb128c7548a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f13e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8035030 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13fc015c50 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f13f80048c0 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13f804d050 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f140c2c3740 10 client.15813 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.823+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564052376620 con 0x5640526332b0 2024-09-12T02:56:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13fc015c50 con 0x5640526332b0 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13f8003c00 con 0x5640526332b0 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 10 client.15813 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 20 client.15813 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x564052650100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 4 client.15813 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640525a4a20 con 0x56405264dd10 2024-09-12T02:56:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.824+0000 7f14037fe640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x564052650100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.825+0000 7f14037fe640 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x564052650100 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2420099cd6d2e6 server_cookie=d1a2112b87a5a098 in_seq=0 out_seq=0 2024-09-12T02:56:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.825+0000 7f13e6ff5640 10 client.15813 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f13e6ff5640 1 -- 172.21.15.46:0/2009257371 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640525a4a20 con 0x56405264dd10 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f13e6ff5640 10 client.15813 handle_command_reply: tid=0 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 client.15813 shutdown 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 msgr2=0x56405263b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f13f8031440 tx=0x7f13f80052f0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 shutdown_connections 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x564052650100 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f13e6ff5640 0 client.15813 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56405263fec0 0x5640526498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x56405263b9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 --2- 172.21.15.46:0/2009257371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564052632990 0x56405263b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 >> 172.21.15.46:0/2009257371 conn(0x56405262f060 msgr2=0x564052634dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 shutdown_connections 2024-09-12T02:56:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 1 -- 172.21.15.46:0/2009257371 wait complete. 2024-09-12T02:56:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.826+0000 7f140c2c3740 20 client.15813 trim_cache size 0 max 16384 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 Processor -- start 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 -- start start 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564052632990 0x5640526559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052655f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14041af3f0 con 0x5640526332b0 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f14041accc0 con 0x564052632990 2024-09-12T02:56:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f140c2c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14041aab10 con 0x56405263fec0 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f14027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f14027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53124/0 (socket says 172.21.15.46:53124) 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f14027fc640 1 -- 172.21.15.46:0/3551011277 learned_addr learned my addr 172.21.15.46:0/3551011277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f14037fe640 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564052632990 0x5640526559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.843+0000 7f14037fe640 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052655f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 -- 172.21.15.46:0/3551011277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 msgr2=0x564052655f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052655f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 -- 172.21.15.46:0/3551011277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564052632990 msgr2=0x5640526559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564052632990 0x5640526559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 -- 172.21.15.46:0/3551011277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564052650470 con 0x5640526332b0 2024-09-12T02:56:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f14027fc640 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f13f8004010 tx=0x7f13f8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e7a0de1a4978bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f13e8ff9640 1 -- 172.21.15.46:0/3551011277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8056070 con 0x5640526332b0 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f13e8ff9640 1 -- 172.21.15.46:0/3551011277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f803f070 con 0x5640526332b0 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.844+0000 7f13e8ff9640 1 -- 172.21.15.46:0/3551011277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8035070 con 0x5640526332b0 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- 172.21.15.46:0/3551011277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 msgr2=0x5640526498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f13f8004010 tx=0x7f13f8009230 comp rx=0 tx=0).stop 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- 172.21.15.46:0/3551011277 shutdown_connections 2024-09-12T02:56:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052655f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5640526332b0 0x5640526498f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3551011277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564052632990 0x5640526559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- 172.21.15.46:0/3551011277 >> 172.21.15.46:0/3551011277 conn(0x56405262f060 msgr2=0x56405263f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- 172.21.15.46:0/3551011277 shutdown_connections 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- 172.21.15.46:0/3551011277 wait complete. 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 -- start start 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.845+0000 7f140c2c3740 1 Processor -- start 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 0x5640526447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052644d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14041ab180 con 0x564052632990 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f14041ac5f0 con 0x5640526332b0 2024-09-12T02:56:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14041ab850 con 0x56405263fec0 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 10 client.0 _ll_register_callbacks cb 0x7f13f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53126/0 (socket says 172.21.15.46:53126) 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 -- 172.21.15.46:0/3788424954 learned_addr learned my addr 172.21.15.46:0/3788424954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052644d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14027fc640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 0x5640526447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 -- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 msgr2=0x564052644d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052644d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 -- 172.21.15.46:0/3788424954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 msgr2=0x5640526447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 0x5640526447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 -- 172.21.15.46:0/3788424954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564052645240 con 0x564052632990 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14027fc640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 0x5640526447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f14037fe640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fb0 tx=0x7f13fc0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31137b3827affbd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f13e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc03a060 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f13fc01d3d0 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.846+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f803f070 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13fc041050 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640526b8720 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f803f070 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13fc00f040 con 0x564052632990 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 20 client.15819 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x5640526bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 4 client.15819 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56405259ed70 con 0x56405264dd10 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.847+0000 7f14027fc640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x5640526bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.848+0000 7f14027fc640 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x5640526bab70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a682e7851e52b36f server_cookie=6dff05c84baefe88 in_seq=0 out_seq=0 2024-09-12T02:56:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.848+0000 7f13e6ff5640 10 client.15819 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f13e6ff5640 1 -- 172.21.15.46:0/3788424954 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56405259ed70 con 0x56405264dd10 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f13e6ff5640 10 client.15819 handle_command_reply: tid=0 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 client.15819 shutdown 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 msgr2=0x564052649450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009fb0 tx=0x7f13fc0098a0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 shutdown_connections 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56405264dd10 0x5640526bab70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56405263fec0 0x564052644d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5640526332b0 0x5640526447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f13e6ff5640 0 client.15819 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 --2- 172.21.15.46:0/3788424954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564052632990 0x564052649450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 >> 172.21.15.46:0/3788424954 conn(0x56405262f060 msgr2=0x5640526344b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 shutdown_connections 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 1 -- 172.21.15.46:0/3788424954 wait complete. 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.856+0000 7f140c2c3740 20 client.15819 trim_cache size 0 max 16384 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:56:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:56:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 msgr2=0x7f13ec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 0x7f13ec0a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ccf90 tx=0x7f13f40d3640 comp rx=0 tx=0).stop 2024-09-12T02:56:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 msgr2=0x7f1404197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9f0 tx=0x7f13f400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 shutdown_connections 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f13ec0a2e20 0x7f13ec0a52e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14041069c0 0x7f140419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1404101350 0x7f14041976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 --2- 172.21.15.46:0/1964026508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14041009b0 0x7f1404197190 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 >> 172.21.15.46:0/1964026508 conn(0x7f1404077990 msgr2=0x7f1404103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 shutdown_connections 2024-09-12T02:56:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:07.857+0000 7f140952a640 1 -- 172.21.15.46:0/1964026508 wait complete. 2024-09-12T02:56:08.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:08 smithi196 ceph-mon[34587]: pgmap v181: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:08.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:08 smithi046 ceph-mon[30951]: pgmap v181: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:08 smithi145 ceph-mon[34674]: pgmap v181: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:10.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:10 smithi196 ceph-mon[34587]: pgmap v182: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:10 smithi046 ceph-mon[30951]: pgmap v182: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:10 smithi145 ceph-mon[34674]: pgmap v182: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:12.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:12 smithi196 ceph-mon[34587]: pgmap v183: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:12 smithi046 ceph-mon[30951]: pgmap v183: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:12 smithi145 ceph-mon[34674]: pgmap v183: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-09-12T02:56:14.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:14 smithi196 ceph-mon[34587]: pgmap v184: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:14 smithi046 ceph-mon[30951]: pgmap v184: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:14 smithi145 ceph-mon[34674]: pgmap v184: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 38 op/s 2024-09-12T02:56:16.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:16 smithi196 ceph-mon[34587]: pgmap v185: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:56:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:16 smithi046 ceph-mon[30951]: pgmap v185: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:56:16.613 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:16 smithi145 ceph-mon[34674]: pgmap v185: 129 pgs: 129 active+clean; 7.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:56:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:18 smithi196 ceph-mon[34587]: pgmap v186: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:18.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:18 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:18 smithi046 ceph-mon[30951]: pgmap v186: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:18 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:18 smithi145 ceph-mon[34674]: pgmap v186: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:18 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:19.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:56:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:56:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:56:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:20 smithi046 ceph-mon[30951]: pgmap v187: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:20.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:20 smithi145 ceph-mon[34674]: pgmap v187: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:20 smithi196 ceph-mon[34587]: pgmap v187: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/748080022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a681069c0 msgr2=0x7f6a68106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/748080022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a681069c0 0x7f6a68106da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009930 tx=0x7f6a5802a420 comp rx=0 tx=0).stop 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/748080022 shutdown_connections 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/748080022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a68101350 0x7f6a6810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/748080022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681009b0 0x7f6a68100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/748080022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a681069c0 0x7f6a68106da0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/748080022 >> 172.21.15.46:0/748080022 conn(0x7f6a68077990 msgr2=0x7f6a680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/748080022 shutdown_connections 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/748080022 wait complete. 2024-09-12T02:56:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.953+0000 7f6a6ccaf640 1 Processor -- start 2024-09-12T02:56:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 -- start start 2024-09-12T02:56:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 0x7f6a68102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681069c0 0x7f6a68103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68110d00 con 0x7f6a68101350 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68110b80 con 0x7f6a681069c0 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6ccaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a68110e80 con 0x7f6a681009b0 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a65d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 0x7f6a68102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:60826/0 (socket says 172.21.15.46:60826) 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 -- 172.21.15.46:0/2637342623 learned_addr learned my addr 172.21.15.46:0/2637342623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681069c0 0x7f6a68103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 -- 172.21.15.46:0/2637342623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681069c0 msgr2=0x7f6a68103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681069c0 0x7f6a68103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 -- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 msgr2=0x7f6a68102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 0x7f6a68102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a6659c640 1 -- 172.21.15.46:0/2637342623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a681a6740 con 0x7f6a681009b0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.954+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 0x7f6a68102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005ab0 tx=0x7f6a58003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a577fe640 1 -- 172.21.15.46:0/2637342623 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5803f070 con 0x7f6a681009b0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a681a69d0 con 0x7f6a681009b0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a577fe640 1 -- 172.21.15.46:0/2637342623 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a58031d60 con 0x7f6a681009b0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a577fe640 1 -- 172.21.15.46:0/2637342623 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58035030 con 0x7f6a681009b0 2024-09-12T02:56:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.955+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a681a6f10 con 0x7f6a681009b0 2024-09-12T02:56:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 Processor -- start 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 -- start start 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b6af0 0x5592197b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55921971b2f0 con 0x5592197c1cb0 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5592196c3270 con 0x5592197b6af0 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592197172b0 con 0x5592197b7410 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a577fe640 1 -- 172.21.15.46:0/2637342623 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6a58042070 con 0x7f6a681009b0 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a65d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a65d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56252/0 (socket says 172.21.15.46:56252) 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a65d9b640 1 -- 172.21.15.46:0/1136085969 learned_addr learned my addr 172.21.15.46:0/1136085969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.958+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b6af0 0x5592197b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 -- 172.21.15.46:0/1136085969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 msgr2=0x5592197c1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 -- 172.21.15.46:0/1136085969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b6af0 msgr2=0x5592197b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b6af0 0x5592197b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 -- 172.21.15.46:0/1136085969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592196c2340 con 0x5592197c1cb0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c010480 tx=0x7f6a5c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea606bdc17148424 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a54ff9640 1 -- 172.21.15.46:0/1136085969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011760 con 0x5592197c1cb0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a54ff9640 1 -- 172.21.15.46:0/1136085969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5c011db0 con 0x5592197c1cb0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a54ff9640 1 -- 172.21.15.46:0/1136085969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00ee50 con 0x5592197c1cb0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a577fe640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 0x7f6a500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 0x7f6a500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.959+0000 7f6a577fe640 1 -- 172.21.15.46:0/2637342623 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6a580eff10 con 0x7f6a681009b0 2024-09-12T02:56:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 0x7f6a500a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005fd0 tx=0x7f6a60009600 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- 172.21.15.46:0/1136085969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 msgr2=0x5592197c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c010480 tx=0x7f6a5c010940 comp rx=0 tx=0).stop 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- 172.21.15.46:0/1136085969 shutdown_connections 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197c1cb0 0x5592197c40a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197c1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/1136085969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b6af0 0x5592197b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- 172.21.15.46:0/1136085969 >> 172.21.15.46:0/1136085969 conn(0x5592197b3060 msgr2=0x5592197b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- 172.21.15.46:0/1136085969 shutdown_connections 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- 172.21.15.46:0/1136085969 wait complete. 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 -- start start 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.960+0000 7f6a6fa42740 1 Processor -- start 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5592197c4880 con 0x5592197b6af0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5592196a4290 con 0x5592197c1cb0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592197c4a00 con 0x5592197b7410 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6fa42740 10 client.0 _ll_register_callbacks cb 0x5592196d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56268/0 (socket says 172.21.15.46:56268) 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 -- 172.21.15.46:0/2868700981 learned_addr learned my addr 172.21.15.46:0/2868700981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 -- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 msgr2=0x5592197bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 -- 172.21.15.46:0/2868700981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 msgr2=0x5592197bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 -- 172.21.15.46:0/2868700981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55921942e210 con 0x5592197b6af0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a6659c640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ed360 tx=0x7f6a580b8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b846ff4f99482c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.961+0000 7f6a4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58049050 con 0x5592197b6af0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6a580ed580 con 0x5592197b6af0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5c011db0 con 0x5592197b6af0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a580bbae0 con 0x5592197b6af0 2024-09-12T02:56:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 10 client.15831 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592194fa620 con 0x5592197b6af0 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5c011db0 con 0x5592197b6af0 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6a5803b580 con 0x5592197b6af0 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 10 client.15831 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 20 client.15831 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5592197d1c10 0x5592197d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 4 client.15831 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559219728a20 con 0x5592197d1c10 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.962+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5592197d1c10 0x5592197d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.963+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5592197d1c10 0x5592197d4000 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d049a27fe373df24 server_cookie=ec4a6887e10393d3 in_seq=0 out_seq=0 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.963+0000 7f6a4dffb640 10 client.15831 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.963+0000 7f6a4dffb640 1 -- 172.21.15.46:0/2868700981 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559219728a20 con 0x5592197d1c10 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.963+0000 7f6a4dffb640 10 client.15831 handle_command_reply: tid=0 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 client.15831 shutdown 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 msgr2=0x5592197bec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ed360 tx=0x7f6a580b8420 comp rx=0 tx=0).stop 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 shutdown_connections 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5592197d1c10 0x5592197d4000 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/2868700981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bec10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 >> 172.21.15.46:0/2868700981 conn(0x5592197b3060 msgr2=0x5592197b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a4dffb640 0 client.15831 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 shutdown_connections 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 1 -- 172.21.15.46:0/2868700981 wait complete. 2024-09-12T02:56:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.964+0000 7f6a6fa42740 20 client.15831 trim_cache size 0 max 16384 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 Processor -- start 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 -- start start 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b7410 0x5592197d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 0x5592197d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681b3780 con 0x5592197b6af0 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681b1050 con 0x5592197b7410 2024-09-12T02:56:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681aeea0 con 0x5592197c1cb0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.980+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56282/0 (socket says 172.21.15.46:56282) 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 -- 172.21.15.46:0/4214327194 learned_addr learned my addr 172.21.15.46:0/4214327194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b7410 0x5592197d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 0x5592197d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 -- 172.21.15.46:0/4214327194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 msgr2=0x5592197d9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 0x5592197d9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 -- 172.21.15.46:0/4214327194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b7410 msgr2=0x5592197d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b7410 0x5592197d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 -- 172.21.15.46:0/4214327194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592197bdba0 con 0x5592197b6af0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 0x5592197d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ed3b0 tx=0x7f6a58009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a8d22b925a5f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a4ffff640 1 -- 172.21.15.46:0/4214327194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58049070 con 0x5592197b6af0 2024-09-12T02:56:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a4ffff640 1 -- 172.21.15.46:0/4214327194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a580c0480 con 0x5592197b6af0 2024-09-12T02:56:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.981+0000 7f6a4ffff640 1 -- 172.21.15.46:0/4214327194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a580c4d40 con 0x5592197b6af0 2024-09-12T02:56:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4214327194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 msgr2=0x5592197bf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ed3b0 tx=0x7f6a58009260 comp rx=0 tx=0).stop 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4214327194 shutdown_connections 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197c1cb0 0x5592197d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197b7410 0x5592197d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4214327194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bf4c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4214327194 >> 172.21.15.46:0/4214327194 conn(0x5592197b3060 msgr2=0x5592197c0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4214327194 shutdown_connections 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.982+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4214327194 wait complete. 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 -- start start 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 Processor -- start 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681af510 con 0x5592197b6af0 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681b0980 con 0x5592197c1cb0 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a681afbe0 con 0x5592197b7410 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6fa42740 10 client.0 _ll_register_callbacks cb 0x7f6a58009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56284/0 (socket says 172.21.15.46:56284) 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 -- 172.21.15.46:0/4275305660 learned_addr learned my addr 172.21.15.46:0/4275305660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 -- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 msgr2=0x5592197bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 -- 172.21.15.46:0/4275305660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 msgr2=0x5592197bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a6659c640 1 -- 172.21.15.46:0/4275305660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55921983b8b0 con 0x5592197b6af0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.983+0000 7f6a66d9d640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6659c640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a580f02c0 tx=0x7f6a580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8415b6774add0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58049070 con 0x5592197b6af0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f6a580318b0 con 0x5592197b6af0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a580c0480 con 0x5592197b6af0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a580bbf50 con 0x5592197b6af0 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 10 client.15837 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55921983b8b0 con 0x5592197b6af0 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a580c0480 con 0x5592197b6af0 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6a580b7ed0 con 0x5592197b6af0 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 20 client.15837 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55921983cd20 0x55921983f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 4 client.15837 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:56:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.984+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559219722d70 con 0x55921983cd20 2024-09-12T02:56:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.985+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55921983cd20 0x55921983f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.985+0000 7f6a4dffb640 10 client.15837 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.985+0000 7f6a65d9b640 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55921983cd20 0x55921983f110 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a57cea7ac2d6a3 server_cookie=c90311ccad02294a in_seq=0 out_seq=0 2024-09-12T02:56:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a4dffb640 1 -- 172.21.15.46:0/4275305660 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559219722d70 con 0x55921983cd20 2024-09-12T02:56:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a4dffb640 10 client.15837 handle_command_reply: tid=0 2024-09-12T02:56:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 client.15837 shutdown 2024-09-12T02:56:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 msgr2=0x5592197bbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a580f02c0 tx=0x7f6a580045e0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 shutdown_connections 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55921983cd20 0x55921983f110 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592197c1cb0 0x5592197bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592197b7410 0x5592197bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 --2- 172.21.15.46:0/4275305660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592197b6af0 0x5592197bbb50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a4dffb640 0 client.15837 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 >> 172.21.15.46:0/4275305660 conn(0x5592197b3060 msgr2=0x5592197b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 shutdown_connections 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 1 -- 172.21.15.46:0/4275305660 wait complete. 2024-09-12T02:56:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.993+0000 7f6a6fa42740 20 client.15837 trim_cache size 0 max 16384 2024-09-12T02:56:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:56:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:56:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:56:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 msgr2=0x7f6a500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 0x7f6a500a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005fd0 tx=0x7f6a60009600 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 msgr2=0x7f6a68102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005ab0 tx=0x7f6a58003cd0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 shutdown_connections 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6a500a2da0 0x7f6a500a5260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a681069c0 0x7f6a68103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a68101350 0x7f6a68102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 --2- 172.21.15.46:0/2637342623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a681009b0 0x7f6a68102580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 >> 172.21.15.46:0/2637342623 conn(0x7f6a68077990 msgr2=0x7f6a68069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 shutdown_connections 2024-09-12T02:56:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:20.994+0000 7f6a6ccaf640 1 -- 172.21.15.46:0/2637342623 wait complete. 2024-09-12T02:56:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:22 smithi046 ceph-mon[30951]: pgmap v188: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:22 smithi145 ceph-mon[34674]: pgmap v188: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:22 smithi196 ceph-mon[34587]: pgmap v188: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:24.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:24 smithi196 ceph-mon[34587]: pgmap v189: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:56:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:24 smithi046 ceph-mon[30951]: pgmap v189: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:56:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:24 smithi145 ceph-mon[34674]: pgmap v189: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:56:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:26 smithi046 ceph-mon[30951]: pgmap v190: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:56:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:26 smithi145 ceph-mon[34674]: pgmap v190: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:56:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:26 smithi196 ceph-mon[34587]: pgmap v190: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:56:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:28 smithi046 ceph-mon[30951]: pgmap v191: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:28 smithi145 ceph-mon[34674]: pgmap v191: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:28 smithi196 ceph-mon[34587]: pgmap v191: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:30 smithi046 ceph-mon[30951]: pgmap v192: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:30 smithi145 ceph-mon[34674]: pgmap v192: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:30 smithi196 ceph-mon[34587]: pgmap v192: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:32.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:32 smithi046 ceph-mon[30951]: pgmap v193: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:32 smithi145 ceph-mon[34674]: pgmap v193: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:32 smithi196 ceph-mon[34587]: pgmap v193: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 -- 172.21.15.46:0/3533262231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 msgr2=0x7fc66010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 --2- 172.21.15.46:0/3533262231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc66010e9a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc658009cd0 tx=0x7fc65802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 -- 172.21.15.46:0/3533262231 shutdown_connections 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 --2- 172.21.15.46:0/3533262231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc66010e9a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 --2- 172.21.15.46:0/3533262231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660101ad0 0x7fc660101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 --2- 172.21.15.46:0/3533262231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660107b70 0x7fc660107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 -- 172.21.15.46:0/3533262231 >> 172.21.15.46:0/3533262231 conn(0x7fc6600fd7d0 msgr2=0x7fc6600ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 -- 172.21.15.46:0/3533262231 shutdown_connections 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.087+0000 7fc66645e640 1 -- 172.21.15.46:0/3533262231 wait complete. 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 Processor -- start 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 -- start start 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 0x7fc660198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660107b70 0x7fc66019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111dd0 con 0x7fc660102470 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111c50 con 0x7fc660101ad0 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc66645e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111f50 con 0x7fc660107b70 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc65f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 0x7fc660198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc65f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57464/0 (socket says 172.21.15.46:57464) 2024-09-12T02:56:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.088+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1880814302 learned_addr learned my addr 172.21.15.46:0/1880814302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc6649fb640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660107b70 0x7fc66019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1880814302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660107b70 msgr2=0x7fc66019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660107b70 0x7fc66019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1880814302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 msgr2=0x7fc660198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 0x7fc660198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1880814302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66019d270 con 0x7fc660102470 2024-09-12T02:56:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65ffff640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 0x7fc660198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc65400d950 tx=0x7fc65400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65d7fa640 1 -- 172.21.15.46:0/1880814302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654017070 con 0x7fc660102470 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65d7fa640 1 -- 172.21.15.46:0/1880814302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc654009c30 con 0x7fc660102470 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66019d560 con 0x7fc660102470 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc65d7fa640 1 -- 172.21.15.46:0/1880814302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65400f620 con 0x7fc660102470 2024-09-12T02:56:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.089+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6601a4df0 con 0x7fc660102470 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.092+0000 7fc65d7fa640 1 -- 172.21.15.46:0/1880814302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc6540109a0 con 0x7fc660102470 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.092+0000 7fc6691f8740 1 Processor -- start 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.092+0000 7fc6691f8740 1 -- start start 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de57faf0 0x5604de57fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de58d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65d7fa640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 0x7fc6480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604de4e42f0 con 0x5604de58acb0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5604de48c270 con 0x5604de580410 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6691f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5604de4e02b0 con 0x5604de57faf0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6649fb640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 0x7fc6480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de58d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc6649fb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de57faf0 0x5604de57fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65d7fa640 1 -- 172.21.15.46:0/1880814302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc6540cf3e0 con 0x7fc660102470 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:35872/0 (socket says 172.21.15.46:35872) 2024-09-12T02:56:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65ffff640 1 -- 172.21.15.46:0/2598006657 learned_addr learned my addr 172.21.15.46:0/2598006657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 -- 172.21.15.46:0/2598006657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de57faf0 msgr2=0x5604de57fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de57faf0 0x5604de57fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 -- 172.21.15.46:0/2598006657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 msgr2=0x5604de58a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65f7fe640 1 -- 172.21.15.46:0/2598006657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604de48b340 con 0x5604de58acb0 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.093+0000 7fc65ffff640 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.094+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de58d0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc6540cc7d0 tx=0x7fc65401e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6184e26b307a650 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.094+0000 7fc6649fb640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 0x7fc6480a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc658009430 tx=0x7fc658009800 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.094+0000 7fc6467fc640 1 -- 172.21.15.46:0/2598006657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654020040 con 0x5604de58acb0 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.094+0000 7fc6467fc640 1 -- 172.21.15.46:0/2598006657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc65409bca0 con 0x5604de58acb0 2024-09-12T02:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.094+0000 7fc6467fc640 1 -- 172.21.15.46:0/2598006657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540071c0 con 0x5604de58acb0 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- 172.21.15.46:0/2598006657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 msgr2=0x5604de58d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de58d0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc6540cc7d0 tx=0x7fc65401e770 comp rx=0 tx=0).stop 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- 172.21.15.46:0/2598006657 shutdown_connections 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de58d0a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2598006657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de57faf0 0x5604de57fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- 172.21.15.46:0/2598006657 >> 172.21.15.46:0/2598006657 conn(0x5604de57c060 msgr2=0x5604de57c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- 172.21.15.46:0/2598006657 shutdown_connections 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- 172.21.15.46:0/2598006657 wait complete. 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 -- start start 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.095+0000 7fc6691f8740 1 Processor -- start 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de57faf0 0x5604de587b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de5880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604de58d880 con 0x5604de58acb0 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5604de46d290 con 0x5604de57faf0 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5604de58da00 con 0x5604de580410 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6691f8740 10 client.0 _ll_register_callbacks cb 0x5604de49e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57484/0 (socket says 172.21.15.46:57484) 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 -- 172.21.15.46:0/779065435 learned_addr learned my addr 172.21.15.46:0/779065435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de5880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc65ffff640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de57faf0 0x5604de587b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 -- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 msgr2=0x5604de5880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de5880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 -- 172.21.15.46:0/779065435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de57faf0 msgr2=0x5604de587b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de57faf0 0x5604de587b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 -- 172.21.15.46:0/779065435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604de1f7210 con 0x5604de58acb0 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de5880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc6649fb640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc658031660 tx=0x7fc65804a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18308f29574bf8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc63f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6580043d0 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65409bca0 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.096+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc658004570 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc658004770 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 10 client.15849 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604de2c3620 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc65409bca0 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc65804a070 con 0x5604de58acb0 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 10 client.15849 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 20 client.15849 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de59abf0 0x5604de59cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 4 client.15849 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.097+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604de4f1a20 con 0x5604de59abf0 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.098+0000 7fc65ffff640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de59abf0 0x5604de59cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.098+0000 7fc63f7fe640 10 client.15849 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.098+0000 7fc65ffff640 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de59abf0 0x5604de59cfe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3053a2e4c488fa11 server_cookie=18381bc7366f78aa in_seq=0 out_seq=0 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc63f7fe640 1 -- 172.21.15.46:0/779065435 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604de4f1a20 con 0x5604de59abf0 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc63f7fe640 10 client.15849 handle_command_reply: tid=0 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 client.15849 shutdown 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 msgr2=0x5604de5885e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc658031660 tx=0x7fc65804a000 comp rx=0 tx=0).stop 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 shutdown_connections 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de59abf0 0x5604de59cfe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de58acb0 0x5604de5885e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de5880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc63f7fe640 0 client.15849 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 --2- 172.21.15.46:0/779065435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de57faf0 0x5604de587b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 >> 172.21.15.46:0/779065435 conn(0x5604de57c060 msgr2=0x5604de581dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 shutdown_connections 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 1 -- 172.21.15.46:0/779065435 wait complete. 2024-09-12T02:56:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.099+0000 7fc6691f8740 20 client.15849 trim_cache size 0 max 16384 2024-09-12T02:56:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 Processor -- start 2024-09-12T02:56:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 -- start start 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de588410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de58acb0 0x5604de5a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.116+0000 7fc6691f8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601af510 con 0x5604de57faf0 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc6691f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601acde0 con 0x5604de58acb0 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc6691f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601aac30 con 0x5604de580410 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57486/0 (socket says 172.21.15.46:57486) 2024-09-12T02:56:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 -- 172.21.15.46:0/2724262108 learned_addr learned my addr 172.21.15.46:0/2724262108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de588410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc6649fb640 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de58acb0 0x5604de5a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 -- 172.21.15.46:0/2724262108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 msgr2=0x5604de588410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de588410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 -- 172.21.15.46:0/2724262108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de58acb0 msgr2=0x5604de5a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de58acb0 0x5604de5a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 -- 172.21.15.46:0/2724262108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604de59d660 con 0x5604de57faf0 2024-09-12T02:56:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc65ffff640 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc650011930 tx=0x7fc65000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3745683c7e9431 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc645ffb640 1 -- 172.21.15.46:0/2724262108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650004220 con 0x5604de57faf0 2024-09-12T02:56:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.117+0000 7fc645ffb640 1 -- 172.21.15.46:0/2724262108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6500043c0 con 0x5604de57faf0 2024-09-12T02:56:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.118+0000 7fc645ffb640 1 -- 172.21.15.46:0/2724262108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650004a00 con 0x5604de57faf0 2024-09-12T02:56:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.118+0000 7fc6691f8740 1 -- 172.21.15.46:0/2724262108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 msgr2=0x5604de589e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.118+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc650011930 tx=0x7fc65000b390 comp rx=0 tx=0).stop 2024-09-12T02:56:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 -- 172.21.15.46:0/2724262108 shutdown_connections 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de58acb0 0x5604de5a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de580410 0x5604de588410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 --2- 172.21.15.46:0/2724262108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de589e90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 -- 172.21.15.46:0/2724262108 >> 172.21.15.46:0/2724262108 conn(0x5604de57c060 msgr2=0x5604de57b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 -- 172.21.15.46:0/2724262108 shutdown_connections 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 -- 172.21.15.46:0/2724262108 wait complete. 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 -- start start 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.119+0000 7fc6691f8740 1 Processor -- start 2024-09-12T02:56:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de58acb0 0x5604de6048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ab2a0 con 0x5604de57faf0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ac710 con 0x5604de580410 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ab970 con 0x5604de58acb0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6691f8740 10 client.0 _ll_register_callbacks cb 0x7fc658031b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57502/0 (socket says 172.21.15.46:57502) 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 -- 172.21.15.46:0/3274870858 learned_addr learned my addr 172.21.15.46:0/3274870858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc6649fb640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de58acb0 0x5604de6048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 -- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de58acb0 msgr2=0x5604de6048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de58acb0 0x5604de6048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 -- 172.21.15.46:0/3274870858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 msgr2=0x5604de58f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.120+0000 7fc65ffff640 1 -- 172.21.15.46:0/3274870858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604de57be90 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc65ffff640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc65000bcc0 tx=0x7fc6500116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8d9df376cb87a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc63f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650031020 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7fc650005220 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6500043c0 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc65003c050 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc6691f8740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.121+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604de605ef0 con 0x5604de57faf0 2024-09-12T02:56:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6500043c0 con 0x5604de57faf0 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc65002c050 con 0x5604de57faf0 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 20 client.15855 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de6065a0 0x5604de6089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 4 client.15855 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.122+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604de4ebd70 con 0x5604de6065a0 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.123+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de6065a0 0x5604de6089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.123+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de6065a0 0x5604de6089c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f309cb0a467f8e server_cookie=a6894a80e4d202a8 in_seq=0 out_seq=0 2024-09-12T02:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.123+0000 7fc63f7fe640 10 client.15855 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc63f7fe640 1 -- 172.21.15.46:0/3274870858 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604de4ebd70 con 0x5604de6065a0 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc63f7fe640 10 client.15855 handle_command_reply: tid=0 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 client.15855 shutdown 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 msgr2=0x5604de58f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc65000bcc0 tx=0x7fc6500116f0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 shutdown_connections 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5604de6065a0 0x5604de6089c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604de58acb0 0x5604de6048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5604de580410 0x5604de58f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 --2- 172.21.15.46:0/3274870858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604de57faf0 0x5604de58f160 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 >> 172.21.15.46:0/3274870858 conn(0x5604de57c060 msgr2=0x5604de57a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.131+0000 7fc63f7fe640 0 client.15855 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 shutdown_connections 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc6691f8740 1 -- 172.21.15.46:0/3274870858 wait complete. 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc6691f8740 20 client.15855 trim_cache size 0 max 16384 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:56:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:56:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 msgr2=0x7fc6480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 0x7fc6480a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc658009430 tx=0x7fc658009800 comp rx=0 tx=0).stop 2024-09-12T02:56:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 msgr2=0x7fc6601987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc65400d950 tx=0x7fc65400de10 comp rx=0 tx=0).stop 2024-09-12T02:56:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 shutdown_connections 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc6480a2da0 0x7fc6480a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660107b70 0x7fc66019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc660102470 0x7fc6601987c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 --2- 172.21.15.46:0/1880814302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc660101ad0 0x7fc660198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.132+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 >> 172.21.15.46:0/1880814302 conn(0x7fc6600fd7d0 msgr2=0x7fc6600fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.133+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 shutdown_connections 2024-09-12T02:56:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:34.133+0000 7fc66645e640 1 -- 172.21.15.46:0/1880814302 wait complete. 2024-09-12T02:56:34.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:34 smithi046 ceph-mon[30951]: pgmap v194: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:34 smithi145 ceph-mon[34674]: pgmap v194: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:34 smithi196 ceph-mon[34587]: pgmap v194: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:36 smithi046 ceph-mon[30951]: pgmap v195: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:36 smithi145 ceph-mon[34674]: pgmap v195: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:36 smithi196 ceph-mon[34587]: pgmap v195: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:38.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:38 smithi046 ceph-mon[30951]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:38 smithi145 ceph-mon[34674]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:38 smithi196 ceph-mon[34587]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:40.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:40 smithi046 ceph-mon[30951]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:40.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:40 smithi145 ceph-mon[34674]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:40 smithi196 ceph-mon[34587]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:42.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:42 smithi046 ceph-mon[30951]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:42 smithi145 ceph-mon[34674]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:42 smithi196 ceph-mon[34587]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:44.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:44 smithi046 ceph-mon[30951]: pgmap v199: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:44 smithi145 ceph-mon[34674]: pgmap v199: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:44.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:44 smithi196 ceph-mon[34587]: pgmap v199: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:46.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:46 smithi046 ceph-mon[30951]: pgmap v200: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:46 smithi145 ceph-mon[34674]: pgmap v200: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:46 smithi196 ceph-mon[34587]: pgmap v200: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 -- 172.21.15.46:0/137313974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f0101350 msgr2=0x7f17f010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 --2- 172.21.15.46:0/137313974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f0101350 0x7f17f010d880 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009cd0 tx=0x7f17e802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 -- 172.21.15.46:0/137313974 shutdown_connections 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 --2- 172.21.15.46:0/137313974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f0101350 0x7f17f010d880 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 --2- 172.21.15.46:0/137313974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f01009b0 0x7f17f0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 --2- 172.21.15.46:0/137313974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01069c0 0x7f17f0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 -- 172.21.15.46:0/137313974 >> 172.21.15.46:0/137313974 conn(0x7f17f0077990 msgr2=0x7f17f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 -- 172.21.15.46:0/137313974 shutdown_connections 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.231+0000 7f17f68ba640 1 -- 172.21.15.46:0/137313974 wait complete. 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 Processor -- start 2024-09-12T02:56:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 -- start start 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01009b0 0x7f17f019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101350 0x7f17f019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0110cb0 con 0x7f17f01069c0 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.232+0000 7f17f68ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0110b30 con 0x7f17f01009b0 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f68ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0110e30 con 0x7f17f0101350 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37400/0 (socket says 172.21.15.46:37400) 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 -- 172.21.15.46:0/3839966636 learned_addr learned my addr 172.21.15.46:0/3839966636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17effff640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01009b0 0x7f17f019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101350 0x7f17f019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 -- 172.21.15.46:0/3839966636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101350 msgr2=0x7f17f019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101350 0x7f17f019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 -- 172.21.15.46:0/3839966636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01009b0 msgr2=0x7f17f019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01009b0 0x7f17f019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 -- 172.21.15.46:0/3839966636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f019f450 con 0x7f17f01069c0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17f4e57640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f17e8033040 tx=0x7f17e8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.233+0000 7f17ed7fa640 1 -- 172.21.15.46:0/3839966636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004040 con 0x7f17f01069c0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.234+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f019f6e0 con 0x7f17f01069c0 2024-09-12T02:56:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.234+0000 7f17ed7fa640 1 -- 172.21.15.46:0/3839966636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e8004690 con 0x7f17f01069c0 2024-09-12T02:56:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.234+0000 7f17ed7fa640 1 -- 172.21.15.46:0/3839966636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8030e10 con 0x7f17f01069c0 2024-09-12T02:56:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.234+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a6fd0 con 0x7f17f01069c0 2024-09-12T02:56:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.236+0000 7f17ed7fa640 1 -- 172.21.15.46:0/3839966636 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f17e80305a0 con 0x7f17f01069c0 2024-09-12T02:56:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.236+0000 7f17f9655740 1 Processor -- start 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.236+0000 7f17f9655740 1 -- start start 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843ee410 0x55e0843f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e0843fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0843522f0 con 0x55e0843edaf0 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e0842fa270 con 0x55e0843f8cb0 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f9655740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e08434e2b0 con 0x55e0843ee410 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ed7fa640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 0x7f17d80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37416/0 (socket says 172.21.15.46:37416) 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3345262747 learned_addr learned my addr 172.21.15.46:0/3345262747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17effff640 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843ee410 0x55e0843f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17f4e57640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 0x7f17d80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e0843fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ed7fa640 1 -- 172.21.15.46:0/3839966636 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f17e802e030 con 0x7f17f01069c0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3345262747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843ee410 msgr2=0x55e0843f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843ee410 0x55e0843f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3345262747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 msgr2=0x55e0843fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e0843fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.237+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3345262747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0842f9340 con 0x55e0843edaf0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.238+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f17e4013620 tx=0x7f17e4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf439eb784005c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.238+0000 7f17f4e57640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 0x7f17d80a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17e80b9460 tx=0x7f17e80b92a0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.238+0000 7f17d67fc640 1 -- 172.21.15.46:0/3345262747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4004050 con 0x55e0843edaf0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.238+0000 7f17d67fc640 1 -- 172.21.15.46:0/3345262747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e40041f0 con 0x55e0843edaf0 2024-09-12T02:56:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.238+0000 7f17d67fc640 1 -- 172.21.15.46:0/3345262747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400f7e0 con 0x55e0843edaf0 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- 172.21.15.46:0/3345262747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 msgr2=0x55e0843eded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f17e4013620 tx=0x7f17e4009830 comp rx=0 tx=0).stop 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- 172.21.15.46:0/3345262747 shutdown_connections 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e0843fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843ee410 0x55e0843f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 --2- 172.21.15.46:0/3345262747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843eded0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- 172.21.15.46:0/3345262747 >> 172.21.15.46:0/3345262747 conn(0x55e0843ea060 msgr2=0x55e0843ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- 172.21.15.46:0/3345262747 shutdown_connections 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- 172.21.15.46:0/3345262747 wait complete. 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 -- start start 2024-09-12T02:56:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.239+0000 7f17f9655740 1 Processor -- start 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e0843f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843f8cb0 0x55e0843f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e0843fb880 con 0x55e0843edaf0 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e0842db290 con 0x55e0843ee410 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e0843fba00 con 0x55e0843f8cb0 2024-09-12T02:56:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17f9655740 10 client.0 _ll_register_callbacks cb 0x55e08430c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37432/0 (socket says 172.21.15.46:37432) 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 -- 172.21.15.46:0/1040970513 learned_addr learned my addr 172.21.15.46:0/1040970513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17effff640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e0843f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843f8cb0 0x55e0843f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 -- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843f8cb0 msgr2=0x55e0843f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843f8cb0 0x55e0843f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 -- 172.21.15.46:0/1040970513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 msgr2=0x55e0843f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e0843f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 -- 172.21.15.46:0/1040970513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e084065210 con 0x55e0843edaf0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17effff640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e0843f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009710 tx=0x7f17e40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac9b69e36c2e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.240+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4023020 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e40041f0 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f17e40035f0 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17e4036050 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e084131620 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e40041f0 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f17e402e020 con 0x55e0843edaf0 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 20 client.15867 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e084408b70 0x55e08440af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 4 client.15867 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e08435fa20 con 0x55e084408b70 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.241+0000 7f17effff640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e084408b70 0x55e08440af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.242+0000 7f17effff640 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e084408b70 0x55e08440af60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f627062e9f756391 server_cookie=1afea15f1a2ee173 in_seq=0 out_seq=0 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.242+0000 7f17cf7fe640 10 client.15867 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17cf7fe640 1 -- 172.21.15.46:0/1040970513 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e08435fa20 con 0x55e084408b70 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17cf7fe640 10 client.15867 handle_command_reply: tid=0 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 client.15867 shutdown 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 msgr2=0x55e0843f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009710 tx=0x7f17e40056d0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 shutdown_connections 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e084408b70 0x55e08440af60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843f8cb0 0x55e0843f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e0843f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 --2- 172.21.15.46:0/1040970513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843edaf0 0x55e0843f5b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 >> 172.21.15.46:0/1040970513 conn(0x55e0843ea060 msgr2=0x55e0843efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17cf7fe640 0 client.15867 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 shutdown_connections 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 1 -- 172.21.15.46:0/1040970513 wait complete. 2024-09-12T02:56:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.243+0000 7f17f9655740 20 client.15867 trim_cache size 0 max 16384 2024-09-12T02:56:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 Processor -- start 2024-09-12T02:56:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 -- start start 2024-09-12T02:56:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e0843f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e084410c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b37d0 con 0x55e0843ee410 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b10a0 con 0x55e0843f8cb0 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17f9655740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01aeef0 con 0x55e0843edaf0 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17effff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17effff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37448/0 (socket says 172.21.15.46:37448) 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.261+0000 7f17effff640 1 -- 172.21.15.46:0/1297247766 learned_addr learned my addr 172.21.15.46:0/1297247766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e0843f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 -- 172.21.15.46:0/1297247766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 msgr2=0x55e0843f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e0843f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 -- 172.21.15.46:0/1297247766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 msgr2=0x55e084410c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e084410c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 -- 172.21.15.46:0/1297247766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0843ed920 con 0x55e0843ee410 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17effff640 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55e0843f5400 tx=0x7f17e000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e2d8849e61cea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17d5ffb640 1 -- 172.21.15.46:0/1297247766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0003e20 con 0x55e0843ee410 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17d5ffb640 1 -- 172.21.15.46:0/1297247766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e00054d0 con 0x55e0843ee410 2024-09-12T02:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.262+0000 7f17d5ffb640 1 -- 172.21.15.46:0/1297247766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00116a0 con 0x55e0843ee410 2024-09-12T02:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- 172.21.15.46:0/1297247766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 msgr2=0x55e0844106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55e0843f5400 tx=0x7f17e000b390 comp rx=0 tx=0).stop 2024-09-12T02:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- 172.21.15.46:0/1297247766 shutdown_connections 2024-09-12T02:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843f8cb0 0x55e084410c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843ee410 0x55e0844106d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 --2- 172.21.15.46:0/1297247766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e0843f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- 172.21.15.46:0/1297247766 >> 172.21.15.46:0/1297247766 conn(0x55e0843ea060 msgr2=0x55e0843f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- 172.21.15.46:0/1297247766 shutdown_connections 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- 172.21.15.46:0/1297247766 wait complete. 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 -- start start 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.263+0000 7f17f9655740 1 Processor -- start 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e084400cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e084401200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01af560 con 0x55e0843f8cb0 2024-09-12T02:56:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b09d0 con 0x55e0843ee410 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01afc30 con 0x55e0843edaf0 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17f9655740 10 client.0 _ll_register_callbacks cb 0x7f17e000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37452/0 (socket says 172.21.15.46:37452) 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3996761789 learned_addr learned my addr 172.21.15.46:0/3996761789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e084400cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17effff640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e084401200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 msgr2=0x55e084400cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e084400cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3996761789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 msgr2=0x55e084401200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e084401200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 -- 172.21.15.46:0/3996761789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0843f2710 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17ef7fe640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f17e40059f0 tx=0x7f17e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad2162f2216d086 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.264+0000 7f17cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4023040 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7f17e40041c0 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e00054d0 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17e40133c0 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e084472e50 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e00054d0 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f17e402e070 con 0x55e0843f8cb0 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:56:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 20 client.15873 populate_metadata read hostname 'smithi046' 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e0844734b0 0x55e0844758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 4 client.15873 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e084359d70 con 0x55e0844734b0 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.265+0000 7f17effff640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e0844734b0 0x55e0844758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.266+0000 7f17cf7fe640 10 client.15873 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.266+0000 7f17effff640 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e0844734b0 0x55e0844758d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ec0cc07ec28c5c server_cookie=2648ffe56c0d1369 in_seq=0 out_seq=0 2024-09-12T02:56:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.273+0000 7f17cf7fe640 1 -- 172.21.15.46:0/3996761789 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e084359d70 con 0x55e0844734b0 2024-09-12T02:56:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.273+0000 7f17cf7fe640 10 client.15873 handle_command_reply: tid=0 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.273+0000 7f17f9655740 1 client.15873 shutdown 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 msgr2=0x55e0843f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f17e40059f0 tx=0x7f17e4009f90 comp rx=0 tx=0).stop 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 shutdown_connections 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55e0844734b0 0x55e0844758d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e0843f8cb0 0x55e0843f21d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17cf7fe640 0 client.15873 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0843ee410 0x55e084401200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 --2- 172.21.15.46:0/3996761789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0843edaf0 0x55e084400cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 >> 172.21.15.46:0/3996761789 conn(0x55e0843ea060 msgr2=0x55e0843ef550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 shutdown_connections 2024-09-12T02:56:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 1 -- 172.21.15.46:0/3996761789 wait complete. 2024-09-12T02:56:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f9655740 20 client.15873 trim_cache size 0 max 16384 2024-09-12T02:56:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:56:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:56:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:56:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 msgr2=0x7f17d80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.274+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 0x7f17d80a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17e80b9460 tx=0x7f17e80b92a0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 msgr2=0x7f17f019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f17e8033040 tx=0x7f17e8003970 comp rx=0 tx=0).stop 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 shutdown_connections 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f17d80a2da0 0x7f17d80a5260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17f01069c0 0x7f17f019ed20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101350 0x7f17f019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 --2- 172.21.15.46:0/3839966636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17f01009b0 0x7f17f019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 >> 172.21.15.46:0/3839966636 conn(0x7f17f0077990 msgr2=0x7f17f00698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 shutdown_connections 2024-09-12T02:56:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:56:47.275+0000 7f17f68ba640 1 -- 172.21.15.46:0/3839966636 wait complete. 2024-09-12T02:56:48.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:48 smithi046 ceph-mon[30951]: pgmap v201: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:48 smithi145 ceph-mon[34674]: pgmap v201: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:48 smithi196 ceph-mon[34587]: pgmap v201: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:50.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:50 smithi046 ceph-mon[30951]: pgmap v202: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:50 smithi145 ceph-mon[34674]: pgmap v202: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:50 smithi196 ceph-mon[34587]: pgmap v202: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:52.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:52 smithi046 ceph-mon[30951]: pgmap v203: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:52 smithi145 ceph-mon[34674]: pgmap v203: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:52 smithi196 ceph-mon[34587]: pgmap v203: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:56:54.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:54 smithi196 ceph-mon[34587]: pgmap v204: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:54 smithi046 ceph-mon[30951]: pgmap v204: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:54 smithi145 ceph-mon[34674]: pgmap v204: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:56.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:56 smithi046 ceph-mon[30951]: pgmap v205: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:56 smithi145 ceph-mon[34674]: pgmap v205: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:56 smithi196 ceph-mon[34587]: pgmap v205: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:56:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:56:58 smithi046 ceph-mon[30951]: pgmap v206: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:56:58 smithi145 ceph-mon[34674]: pgmap v206: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:56:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:56:58 smithi196 ceph-mon[34587]: pgmap v206: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-09-12T02:57:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.373+0000 7ff5635fd640 1 -- 172.21.15.46:0/3606064806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c103640 msgr2=0x7ff55c10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.373+0000 7ff5635fd640 1 --2- 172.21.15.46:0/3606064806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c103640 0x7ff55c10bb20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c02a480 comp rx=0 tx=0).stop 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 -- 172.21.15.46:0/3606064806 shutdown_connections 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 --2- 172.21.15.46:0/3606064806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c10c0f0 0x7ff55c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 --2- 172.21.15.46:0/3606064806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c103640 0x7ff55c10bb20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 --2- 172.21.15.46:0/3606064806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 -- 172.21.15.46:0/3606064806 >> 172.21.15.46:0/3606064806 conn(0x7ff55c077990 msgr2=0x7ff55c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 -- 172.21.15.46:0/3606064806 shutdown_connections 2024-09-12T02:57:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 -- 172.21.15.46:0/3606064806 wait complete. 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.374+0000 7ff5635fd640 1 Processor -- start 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 -- start start 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c103640 0x7ff55c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111d70 con 0x7ff55c10c0f0 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111bf0 con 0x7ff55c102d20 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff5635fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111ef0 con 0x7ff55c103640 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56698/0 (socket says 172.21.15.46:56698) 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 -- 172.21.15.46:0/2801527144 learned_addr learned my addr 172.21.15.46:0/2801527144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff560b98640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c103640 0x7ff55c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561399640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 -- 172.21.15.46:0/2801527144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c103640 msgr2=0x7ff55c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c103640 0x7ff55c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 -- 172.21.15.46:0/2801527144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 msgr2=0x7ff55c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561b9a640 1 -- 172.21.15.46:0/2801527144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c19f440 con 0x7ff55c10c0f0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.375+0000 7ff561399640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff561b9a640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff55400eaa0 tx=0x7ff55400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff5527fc640 1 -- 172.21.15.46:0/2801527144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55400cdb0 con 0x7ff55c10c0f0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff5527fc640 1 -- 172.21.15.46:0/2801527144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff554005020 con 0x7ff55c10c0f0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c19f730 con 0x7ff55c10c0f0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff5527fc640 1 -- 172.21.15.46:0/2801527144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540116d0 con 0x7ff55c10c0f0 2024-09-12T02:57:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.376+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c1a6fc0 con 0x7ff55c10c0f0 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.378+0000 7ff5527fc640 1 -- 172.21.15.46:0/2801527144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff5540051d0 con 0x7ff55c10c0f0 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 Processor -- start 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 -- start start 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a77343f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a77343fcb0 0x55a7734420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a7733992f0 con 0x55a773434af0 2024-09-12T02:57:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a773341270 con 0x55a773435410 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff56647f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7733952b0 con 0x55a77343fcb0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff560b98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.379+0000 7ff560b98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56708/0 (socket says 172.21.15.46:56708) 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 -- 172.21.15.46:0/2159825818 learned_addr learned my addr 172.21.15.46:0/2159825818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff5527fc640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 0x7ff5480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff561399640 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a77343f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a77343fcb0 0x55a7734420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff5527fc640 1 -- 172.21.15.46:0/2801527144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff5540cf8a0 con 0x7ff55c10c0f0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff561b9a640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 0x7ff5480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 -- 172.21.15.46:0/2159825818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a77343fcb0 msgr2=0x55a7734420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a77343fcb0 0x55a7734420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 -- 172.21.15.46:0/2159825818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 msgr2=0x55a77343f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a77343f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 -- 172.21.15.46:0/2159825818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a773340340 con 0x55a773434af0 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff560b98640 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1789b02846a442 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.380+0000 7ff561b9a640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 0x7ff5480a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff55409b760 tx=0x7ff5540cfba0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.381+0000 7ff5477fe640 1 -- 172.21.15.46:0/2159825818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c044020 con 0x55a773434af0 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.381+0000 7ff5477fe640 1 -- 172.21.15.46:0/2159825818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff54c031860 con 0x55a773434af0 2024-09-12T02:57:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.381+0000 7ff5477fe640 1 -- 172.21.15.46:0/2159825818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c035040 con 0x55a773434af0 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.381+0000 7ff56647f740 1 -- 172.21.15.46:0/2159825818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 msgr2=0x55a773434ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.381+0000 7ff56647f740 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c005db0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 -- 172.21.15.46:0/2159825818 shutdown_connections 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a77343fcb0 0x55a7734420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a77343f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 --2- 172.21.15.46:0/2159825818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773434af0 0x55a773434ed0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 -- 172.21.15.46:0/2159825818 >> 172.21.15.46:0/2159825818 conn(0x55a773431060 msgr2=0x55a773431440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 -- 172.21.15.46:0/2159825818 shutdown_connections 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 -- 172.21.15.46:0/2159825818 wait complete. 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 -- start start 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.382+0000 7ff56647f740 1 Processor -- start 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a77343d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a77343d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a773442880 con 0x55a773435410 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a773322290 con 0x55a77343fcb0 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a773442a00 con 0x55a773434af0 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff56647f740 10 client.0 _ll_register_callbacks cb 0x55a773353140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff560b98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff560b98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44970/0 (socket says 172.21.15.46:44970) 2024-09-12T02:57:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff560b98640 1 -- 172.21.15.46:0/3401250219 learned_addr learned my addr 172.21.15.46:0/3401250219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff561399640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a77343d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.383+0000 7ff560b98640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a77343d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 -- 172.21.15.46:0/3401250219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 msgr2=0x55a77343d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a77343d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 -- 172.21.15.46:0/3401250219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 msgr2=0x55a77343d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a77343d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 -- 172.21.15.46:0/3401250219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7730ac210 con 0x55a773434af0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff560b98640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff54c031f60 tx=0x7ff54c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b6592cc45b44ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c044070 con 0x55a773434af0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c031860 con 0x55a773434af0 2024-09-12T02:57:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff54c0055f0 con 0x55a773434af0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff54c035070 con 0x55a773434af0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff56647f740 10 client.25462 fetch_fsmap learned FSMap version 29 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.384+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a773178620 con 0x55a773434af0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff54c031860 con 0x55a773434af0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff54c049040 con 0x55a773434af0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 10 client.25462 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 20 client.25462 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a77344faf0 0x55a773451ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 4 client.25462 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7733a6a20 con 0x55a77344faf0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.385+0000 7ff561399640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a77344faf0 0x55a773451ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.386+0000 7ff544ff9640 10 client.25462 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.386+0000 7ff561399640 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a77344faf0 0x55a773451ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342e4040c0408e9d server_cookie=fa82f858eef527d3 in_seq=0 out_seq=0 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.386+0000 7ff544ff9640 1 -- 172.21.15.46:0/3401250219 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7733a6a20 con 0x55a77344faf0 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.386+0000 7ff544ff9640 10 client.25462 handle_command_reply: tid=0 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 client.25462 shutdown 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 msgr2=0x55a77343cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff54c031f60 tx=0x7ff54c02a480 comp rx=0 tx=0).stop 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 shutdown_connections 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a77344faf0 0x55a773451ee0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a77343d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff544ff9640 0 client.25462 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a77343d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 --2- 172.21.15.46:0/3401250219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343cba0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 >> 172.21.15.46:0/3401250219 conn(0x55a773431060 msgr2=0x55a773436dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 shutdown_connections 2024-09-12T02:57:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 1 -- 172.21.15.46:0/3401250219 wait complete. 2024-09-12T02:57:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.387+0000 7ff56647f740 20 client.25462 trim_cache size 0 max 16384 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 Processor -- start 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 -- start start 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a773454f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 0x55a773455440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1b3810 con 0x55a77343fcb0 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1b10e0 con 0x55a773435410 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff56647f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1aef30 con 0x55a773434af0 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff560b98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 0x55a773455440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff560b98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 0x55a773455440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56722/0 (socket says 172.21.15.46:56722) 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.409+0000 7ff560b98640 1 -- 172.21.15.46:0/2752566761 learned_addr learned my addr 172.21.15.46:0/2752566761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff561399640 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a773454f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 -- 172.21.15.46:0/2752566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 msgr2=0x55a773454f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a773454f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 -- 172.21.15.46:0/2752566761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 msgr2=0x55a773455440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 0x55a773455440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 -- 172.21.15.46:0/2752566761 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a773430e90 con 0x55a773434af0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff560b98640 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff54c032210 tx=0x7ff54c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1934f729e3ca4ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff546ffd640 1 -- 172.21.15.46:0/2752566761 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c044040 con 0x55a773434af0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff546ffd640 1 -- 172.21.15.46:0/2752566761 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff54c031fd0 con 0x55a773434af0 2024-09-12T02:57:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.410+0000 7ff546ffd640 1 -- 172.21.15.46:0/2752566761 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0046c0 con 0x55a773434af0 2024-09-12T02:57:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 -- 172.21.15.46:0/2752566761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 msgr2=0x55a77343d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff54c032210 tx=0x7ff54c031400 comp rx=0 tx=0).stop 2024-09-12T02:57:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 -- 172.21.15.46:0/2752566761 shutdown_connections 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a77343fcb0 0x55a773455440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a773435410 0x55a773454f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 --2- 172.21.15.46:0/2752566761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a77343d450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 -- 172.21.15.46:0/2752566761 >> 172.21.15.46:0/2752566761 conn(0x55a773431060 msgr2=0x55a77343ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 -- 172.21.15.46:0/2752566761 shutdown_connections 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.411+0000 7ff56647f740 1 -- 172.21.15.46:0/2752566761 wait complete. 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 -- start start 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 Processor -- start 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a7734555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a7734528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a773452e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1af5a0 con 0x55a773435410 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.412+0000 7ff56647f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1b0a10 con 0x55a77343fcb0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff56647f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1afc70 con 0x55a773434af0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff56647f740 10 client.0 _ll_register_callbacks cb 0x7ff558008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff561399640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a7734528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a7734555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff561399640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a7734528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56728/0 (socket says 172.21.15.46:56728) 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff561399640 1 -- 172.21.15.46:0/2237523468 learned_addr learned my addr 172.21.15.46:0/2237523468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a773452e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 -- 172.21.15.46:0/2237523468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 msgr2=0x55a773452e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a773452e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 -- 172.21.15.46:0/2237523468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 msgr2=0x55a7734528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a7734528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 -- 172.21.15.46:0/2237523468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a77342c2d0 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff560b98640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a7734555a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009640 tx=0x7ff54c032920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b4778993573a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.413+0000 7ff544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c044030 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c031fd0 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9379+0+0 (secure 0 0 0) 0x7ff54c004220 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff54c041070 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 10 client.25468 fetch_fsmap learned FSMap version 29 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7734ba1d0 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff54c031fd0 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff54c003aa0 con 0x55a773434af0 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 10 client.25468 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:57:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 20 client.25468 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a7734ba650 0x55a7734bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 4 client.25468 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.414+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7733a0d70 con 0x55a7734ba650 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.415+0000 7ff561399640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a7734ba650 0x55a7734bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.415+0000 7ff561399640 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a7734ba650 0x55a7734bca40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5bd8e3f28c6e0 server_cookie=d0bb39e424c1ce80 in_seq=0 out_seq=0 2024-09-12T02:57:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.415+0000 7ff544ff9640 10 client.25468 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.461+0000 7ff544ff9640 1 -- 172.21.15.46:0/2237523468 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7733a0d70 con 0x55a7734ba650 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff544ff9640 10 client.25468 handle_command_reply: tid=0 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 client.25468 shutdown 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 msgr2=0x55a7734555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a7734555a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009640 tx=0x7ff54c032920 comp rx=0 tx=0).stop 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 shutdown_connections 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a7734ba650 0x55a7734bca40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a77343fcb0 0x55a773452e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a773435410 0x55a7734528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 --2- 172.21.15.46:0/2237523468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a773434af0 0x55a7734555a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 >> 172.21.15.46:0/2237523468 conn(0x55a773431060 msgr2=0x55a773436550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff544ff9640 0 client.25468 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 shutdown_connections 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 1 -- 172.21.15.46:0/2237523468 wait complete. 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.462+0000 7ff56647f740 20 client.25468 trim_cache size 0 max 16384 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:57:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:57:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:57:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:57:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 msgr2=0x7ff5480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 0x7ff5480a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff55409b760 tx=0x7ff5540cfba0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 msgr2=0x7ff55c19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff55400eaa0 tx=0x7ff55400ef60 comp rx=0 tx=0).stop 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 shutdown_connections 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff5480a2e20 0x7ff5480a52e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff55c10c0f0 0x7ff55c19ed10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff55c103640 0x7ff55c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 --2- 172.21.15.46:0/2801527144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff55c102d20 0x7ff55c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 >> 172.21.15.46:0/2801527144 conn(0x7ff55c077990 msgr2=0x7ff55c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 shutdown_connections 2024-09-12T02:57:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:00.463+0000 7ff5635fd640 1 -- 172.21.15.46:0/2801527144 wait complete. 2024-09-12T02:57:00.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:00 smithi046 ceph-mon[30951]: pgmap v207: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:00 smithi145 ceph-mon[34674]: pgmap v207: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:00 smithi196 ceph-mon[34587]: pgmap v207: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:02 smithi046 ceph-mon[30951]: pgmap v208: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:02 smithi145 ceph-mon[34674]: pgmap v208: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:02 smithi196 ceph-mon[34587]: pgmap v208: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:03.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:03 smithi046 ceph-mon[30951]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T02:57:03.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:03 smithi145 ceph-mon[34674]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T02:57:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:03 smithi196 ceph-mon[34587]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T02:57:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:04 smithi046 ceph-mon[30951]: pgmap v210: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:04 smithi145 ceph-mon[34674]: pgmap v210: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:04 smithi196 ceph-mon[34587]: pgmap v210: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:06 smithi145 ceph-mon[34674]: pgmap v211: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:06 smithi196 ceph-mon[34587]: pgmap v211: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:06.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:06 smithi046 ceph-mon[30951]: pgmap v211: 129 pgs: 129 active+clean; 8.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-12T02:57:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:08 smithi145 ceph-mon[34674]: pgmap v212: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:08 smithi196 ceph-mon[34587]: pgmap v212: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:08.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:08 smithi046 ceph-mon[30951]: pgmap v212: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:10 smithi145 ceph-mon[34674]: pgmap v213: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:10 smithi145 ceph-mon[34674]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T02:57:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:10 smithi196 ceph-mon[34587]: pgmap v213: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:10 smithi196 ceph-mon[34587]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T02:57:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:10 smithi046 ceph-mon[30951]: pgmap v213: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-12T02:57:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:10 smithi046 ceph-mon[30951]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T02:57:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:11 smithi145 ceph-mon[34674]: pgmap v215: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-12T02:57:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:11 smithi196 ceph-mon[34587]: pgmap v215: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-12T02:57:11.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:11 smithi046 ceph-mon[30951]: pgmap v215: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-12T02:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 -- 172.21.15.46:0/1433563608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840101350 msgr2=0x7f884010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 --2- 172.21.15.46:0/1433563608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840101350 0x7f884010d880 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8830009930 tx=0x7f883002a420 comp rx=0 tx=0).stop 2024-09-12T02:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 -- 172.21.15.46:0/1433563608 shutdown_connections 2024-09-12T02:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 --2- 172.21.15.46:0/1433563608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8840101350 0x7f884010d880 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 --2- 172.21.15.46:0/1433563608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88401009b0 0x7f8840100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 --2- 172.21.15.46:0/1433563608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401069c0 0x7f8840106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 -- 172.21.15.46:0/1433563608 >> 172.21.15.46:0/1433563608 conn(0x7f8840077990 msgr2=0x7f88400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 -- 172.21.15.46:0/1433563608 shutdown_connections 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.563+0000 7f8847d11640 1 -- 172.21.15.46:0/1433563608 wait complete. 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 Processor -- start 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 -- start start 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401009b0 0x7f8840198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 0x7f8840198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8840110d00 con 0x7f88401069c0 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8840110b80 con 0x7f88401009b0 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f8847d11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8840110e80 con 0x7f8840101350 2024-09-12T02:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.564+0000 7f88462ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44010/0 (socket says 172.21.15.46:44010) 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 -- 172.21.15.46:0/1236595143 learned_addr learned my addr 172.21.15.46:0/1236595143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f8845aad640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401009b0 0x7f8840198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88452ac640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 0x7f8840198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 -- 172.21.15.46:0/1236595143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 msgr2=0x7f8840198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 0x7f8840198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 -- 172.21.15.46:0/1236595143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401009b0 msgr2=0x7f8840198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401009b0 0x7f8840198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 -- 172.21.15.46:0/1236595143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884019d220 con 0x7f88401069c0 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88452ac640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 0x7f8840198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f88462ae640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f883002a940 tx=0x7f8830003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f883effd640 1 -- 172.21.15.46:0/1236595143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883003f070 con 0x7f88401069c0 2024-09-12T02:57:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884019d4b0 con 0x7f88401069c0 2024-09-12T02:57:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f883effd640 1 -- 172.21.15.46:0/1236595143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8830003e40 con 0x7f88401069c0 2024-09-12T02:57:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f883effd640 1 -- 172.21.15.46:0/1236595143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830035030 con 0x7f88401069c0 2024-09-12T02:57:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.565+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a4da0 con 0x7f88401069c0 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.568+0000 7f883effd640 1 -- 172.21.15.46:0/1236595143 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8830042070 con 0x7f88401069c0 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.568+0000 7f884aaad740 1 Processor -- start 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 -- start start 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f883effd640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 0x7f88280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c43d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c3972f0 con 0x55ba7c433410 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c33f270 con 0x55ba7c43dcb0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f884aaad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c3932b0 con 0x55ba7c432af0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f88452ac640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 0x7f88280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f883effd640 1 -- 172.21.15.46:0/1236595143 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f88300effa0 con 0x7f88401069c0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f88462ae640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f8845aad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f88452ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c43d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f8845aad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:41684/0 (socket says 172.21.15.46:41684) 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f88462ae640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:58060/0 (socket says 172.21.15.46:58060) 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.569+0000 7f8845aad640 1 -- 172.21.15.46:0/4116610692 learned_addr learned my addr 172.21.15.46:0/4116610692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 -- 172.21.15.46:0/4116610692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 msgr2=0x55ba7c4400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 -- 172.21.15.46:0/4116610692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 msgr2=0x55ba7c43d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c43d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 -- 172.21.15.46:0/4116610692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba7c33e340 con 0x55ba7c432af0 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f88452ac640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 0x7f88280a52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8834005d70 tx=0x7f883400bb70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8845aad640 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88380103d0 tx=0x7f8838010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eaa5cf5aad23939 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8827fff640 1 -- 172.21.15.46:0/4116610692 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838003b70 con 0x55ba7c432af0 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8827fff640 1 -- 172.21.15.46:0/4116610692 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015070 con 0x55ba7c432af0 2024-09-12T02:57:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.570+0000 7f8827fff640 1 -- 172.21.15.46:0/4116610692 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838011820 con 0x55ba7c432af0 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 -- 172.21.15.46:0/4116610692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 msgr2=0x55ba7c432ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88380103d0 tx=0x7f8838010890 comp rx=0 tx=0).stop 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 -- 172.21.15.46:0/4116610692 shutdown_connections 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c43d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 --2- 172.21.15.46:0/4116610692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c432ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 -- 172.21.15.46:0/4116610692 >> 172.21.15.46:0/4116610692 conn(0x55ba7c42f060 msgr2=0x55ba7c42f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 -- 172.21.15.46:0/4116610692 shutdown_connections 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.571+0000 7f884aaad740 1 -- 172.21.15.46:0/4116610692 wait complete. 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 -- start start 2024-09-12T02:57:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 Processor -- start 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c43ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 0x55ba7c43b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c444c90 con 0x55ba7c43dcb0 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c320290 con 0x55ba7c433410 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba7c444e10 con 0x55ba7c432af0 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f884aaad740 10 client.0 _ll_register_callbacks cb 0x55ba7c351140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f88462ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f88462ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44030/0 (socket says 172.21.15.46:44030) 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f88462ae640 1 -- 172.21.15.46:0/505380464 learned_addr learned my addr 172.21.15.46:0/505380464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f8845aad640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c43ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.572+0000 7f88452ac640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 0x55ba7c43b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 -- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 msgr2=0x55ba7c43ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c43ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 -- 172.21.15.46:0/505380464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 msgr2=0x55ba7c43b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 0x55ba7c43b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 -- 172.21.15.46:0/505380464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7c0aa210 con 0x55ba7c43dcb0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88452ac640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 0x55ba7c43b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88462ae640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f88300bc0a0 tx=0x7f88300bbd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81031bdde261973 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88257fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830049070 con 0x55ba7c43dcb0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f8830042cf0 con 0x55ba7c43dcb0 2024-09-12T02:57:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838015070 con 0x55ba7c43dcb0 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8830099550 con 0x55ba7c43dcb0 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f884aaad740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.573+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba7c176620 con 0x55ba7c43dcb0 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015070 con 0x55ba7c43dcb0 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f88300ed5c0 con 0x55ba7c43dcb0 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 20 client.15903 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c451fd0 0x55ba7c454420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 4 client.15903 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba7c3a4a20 con 0x55ba7c451fd0 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.574+0000 7f8845aad640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c451fd0 0x55ba7c454420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f88257fa640 10 client.15903 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f8845aad640 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c451fd0 0x55ba7c454420 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4f0334517d8aaa server_cookie=be4680fc94659080 in_seq=0 out_seq=0 2024-09-12T02:57:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f88257fa640 1 -- 172.21.15.46:0/505380464 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba7c3a4a20 con 0x55ba7c451fd0 2024-09-12T02:57:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f88257fa640 10 client.15903 handle_command_reply: tid=0 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f884aaad740 1 client.15903 shutdown 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 msgr2=0x55ba7c43b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.575+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f88300bc0a0 tx=0x7f88300bbd10 comp rx=0 tx=0).stop 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 shutdown_connections 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c451fd0 0x55ba7c454420 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c43dcb0 0x55ba7c43b5f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c433410 0x55ba7c43b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 --2- 172.21.15.46:0/505380464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c43ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f88257fa640 0 client.15903 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 >> 172.21.15.46:0/505380464 conn(0x55ba7c42f060 msgr2=0x55ba7c434dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 shutdown_connections 2024-09-12T02:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 1 -- 172.21.15.46:0/505380464 wait complete. 2024-09-12T02:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.576+0000 7f884aaad740 20 client.15903 trim_cache size 0 max 16384 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.591+0000 7f884aaad740 1 Processor -- start 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 -- start start 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c433410 0x55ba7c459d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c45a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88401af540 con 0x55ba7c432af0 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88401ace10 con 0x55ba7c43dcb0 2024-09-12T02:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f884aaad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88401aac80 con 0x55ba7c433410 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44038/0 (socket says 172.21.15.46:44038) 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 -- 172.21.15.46:0/1714042979 learned_addr learned my addr 172.21.15.46:0/1714042979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f88462ae640 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c45a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f88452ac640 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c433410 0x55ba7c459d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 -- 172.21.15.46:0/1714042979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c433410 msgr2=0x55ba7c459d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c433410 0x55ba7c459d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.592+0000 7f8845aad640 1 -- 172.21.15.46:0/1714042979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 msgr2=0x55ba7c45a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f8845aad640 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c45a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f8845aad640 1 -- 172.21.15.46:0/1714042979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba7c42ee90 con 0x55ba7c432af0 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f8845aad640 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8838010d90 tx=0x7f8838013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c61abae8f1740e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f88277fe640 1 -- 172.21.15.46:0/1714042979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838004af0 con 0x55ba7c432af0 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f88277fe640 1 -- 172.21.15.46:0/1714042979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015070 con 0x55ba7c432af0 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.593+0000 7f88277fe640 1 -- 172.21.15.46:0/1714042979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380052b0 con 0x55ba7c432af0 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 -- 172.21.15.46:0/1714042979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 msgr2=0x55ba7c43cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8838010d90 tx=0x7f8838013040 comp rx=0 tx=0).stop 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 -- 172.21.15.46:0/1714042979 shutdown_connections 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c45a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c433410 0x55ba7c459d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 --2- 172.21.15.46:0/1714042979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c432af0 0x55ba7c43cae0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 -- 172.21.15.46:0/1714042979 >> 172.21.15.46:0/1714042979 conn(0x55ba7c42f060 msgr2=0x55ba7c43c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 -- 172.21.15.46:0/1714042979 shutdown_connections 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.594+0000 7f884aaad740 1 -- 172.21.15.46:0/1714042979 wait complete. 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 -- start start 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 Processor -- start 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c44a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88401ab2d0 con 0x55ba7c433410 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88401ac740 con 0x55ba7c43dcb0 2024-09-12T02:57:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88401ab9a0 con 0x55ba7c432af0 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.595+0000 7f884aaad740 10 client.0 _ll_register_callbacks cb 0x7f88300b7d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44044/0 (socket says 172.21.15.46:44044) 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 -- 172.21.15.46:0/4015359859 learned_addr learned my addr 172.21.15.46:0/4015359859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88462ae640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 -- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 msgr2=0x55ba7c44a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c44a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 -- 172.21.15.46:0/4015359859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 msgr2=0x55ba7c4461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 -- 172.21.15.46:0/4015359859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7c446a10 con 0x55ba7c433410 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88462ae640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88452ac640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8834014010 tx=0x7f8834005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8beb7047ecc0616 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.596+0000 7f88257fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834004020 con 0x55ba7c433410 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f88340041c0 con 0x55ba7c433410 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838015070 con 0x55ba7c433410 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8834034050 con 0x55ba7c433410 2024-09-12T02:57:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f884aaad740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba7c4bce40 con 0x55ba7c433410 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015070 con 0x55ba7c433410 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8834028030 con 0x55ba7c433410 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f884aaad740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.597+0000 7f884aaad740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.598+0000 7f884aaad740 20 client.15909 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.598+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c4bd640 0x55ba7c4bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.598+0000 7f884aaad740 4 client.15909 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.598+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba7c39ed70 con 0x55ba7c4bd640 2024-09-12T02:57:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.598+0000 7f8845aad640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c4bd640 0x55ba7c4bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.599+0000 7f8845aad640 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c4bd640 0x55ba7c4bfa60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83077a2533ac8fa4 server_cookie=4fecf055d97ad371 in_seq=0 out_seq=0 2024-09-12T02:57:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.599+0000 7f88257fa640 10 client.15909 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.607+0000 7f88257fa640 1 -- 172.21.15.46:0/4015359859 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba7c39ed70 con 0x55ba7c4bd640 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f88257fa640 10 client.15909 handle_command_reply: tid=0 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 client.15909 shutdown 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 msgr2=0x55ba7c445c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8834014010 tx=0x7f8834005ea0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 shutdown_connections 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ba7c4bd640 0x55ba7c4bfa60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ba7c43dcb0 0x55ba7c4461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f88257fa640 0 client.15909 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba7c433410 0x55ba7c445c10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 --2- 172.21.15.46:0/4015359859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba7c432af0 0x55ba7c44a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 >> 172.21.15.46:0/4015359859 conn(0x55ba7c42f060 msgr2=0x55ba7c43b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 shutdown_connections 2024-09-12T02:57:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 1 -- 172.21.15.46:0/4015359859 wait complete. 2024-09-12T02:57:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.608+0000 7f884aaad740 20 client.15909 trim_cache size 0 max 16384 2024-09-12T02:57:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:57:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:57:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:57:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:57:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 msgr2=0x7f88280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 0x7f88280a52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8834005d70 tx=0x7f883400bb70 comp rx=0 tx=0).stop 2024-09-12T02:57:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 msgr2=0x7f884019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f883002a940 tx=0x7f8830003cd0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 shutdown_connections 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88280a2e20 0x7f88280a52e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88401069c0 0x7f884019caf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8840101350 0x7f8840198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 --2- 172.21.15.46:0/1236595143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88401009b0 0x7f8840198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 >> 172.21.15.46:0/1236595143 conn(0x7f8840077990 msgr2=0x7f8840069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 shutdown_connections 2024-09-12T02:57:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:13.609+0000 7f8847d11640 1 -- 172.21.15.46:0/1236595143 wait complete. 2024-09-12T02:57:13.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:13 smithi145 ceph-mon[34674]: pgmap v216: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:13.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:13 smithi196 ceph-mon[34587]: pgmap v216: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:14.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:13 smithi046 ceph-mon[30951]: pgmap v216: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:15.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:15 smithi145 ceph-mon[34674]: pgmap v217: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:15.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:15 smithi196 ceph-mon[34587]: pgmap v217: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:16.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:15 smithi046 ceph-mon[30951]: pgmap v217: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:17 smithi145 ceph-mon[34674]: pgmap v218: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:17 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:17 smithi196 ceph-mon[34587]: pgmap v218: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:17 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:18.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:17 smithi046 ceph-mon[30951]: pgmap v218: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:18.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:17 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:18.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:18 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:19.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:18 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:19.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:18 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:19 smithi196 ceph-mon[34587]: pgmap v219: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:19.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:57:20.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:19 smithi046 ceph-mon[30951]: pgmap v219: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:20.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:20.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:20.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:57:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:19 smithi145 ceph-mon[34674]: pgmap v219: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-12T02:57:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:57:21.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:21 smithi196 ceph-mon[34587]: pgmap v220: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:22.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:21 smithi046 ceph-mon[30951]: pgmap v220: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:22.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:21 smithi145 ceph-mon[34674]: pgmap v220: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-12T02:57:22.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:22 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:23.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:22 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:23.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:22 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:57:23.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:23 smithi196 ceph-mon[34587]: pgmap v221: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:24.011 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:23 smithi145 ceph-mon[34674]: pgmap v221: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:24.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:23 smithi046 ceph-mon[30951]: pgmap v221: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:25.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:25 smithi196 ceph-mon[34587]: pgmap v222: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:57:26.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:25 smithi046 ceph-mon[30951]: pgmap v222: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:57:26.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:25 smithi145 ceph-mon[34674]: pgmap v222: 129 pgs: 129 active+clean; 9.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 -- 172.21.15.46:0/1156122973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 msgr2=0x7f7c1c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 --2- 172.21.15.46:0/1156122973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c10d880 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009cd0 tx=0x7f7c1402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 -- 172.21.15.46:0/1156122973 shutdown_connections 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 --2- 172.21.15.46:0/1156122973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c10d880 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 --2- 172.21.15.46:0/1156122973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 --2- 172.21.15.46:0/1156122973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 -- 172.21.15.46:0/1156122973 >> 172.21.15.46:0/1156122973 conn(0x7f7c1c077990 msgr2=0x7f7c1c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 -- 172.21.15.46:0/1156122973 shutdown_connections 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 -- 172.21.15.46:0/1156122973 wait complete. 2024-09-12T02:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.707+0000 7f7c242ac640 1 Processor -- start 2024-09-12T02:57:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 -- start start 2024-09-12T02:57:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c110d00 con 0x7f7c1c101350 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c110b80 con 0x7f7c1c1069c0 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c242ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c110e80 con 0x7f7c1c1009b0 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59244/0 (socket says 172.21.15.46:59244) 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 -- 172.21.15.46:0/4197667989 learned_addr learned my addr 172.21.15.46:0/4197667989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c22048640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c22849640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 -- 172.21.15.46:0/4197667989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 msgr2=0x7f7c1c102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c102580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 -- 172.21.15.46:0/4197667989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 msgr2=0x7f7c1c103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 -- 172.21.15.46:0/4197667989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c1a6790 con 0x7f7c1c101350 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c22048640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c22849640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.708+0000 7f7c21847640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c890 tx=0x7f7c0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.709+0000 7f7c137fe640 1 -- 172.21.15.46:0/4197667989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c004220 con 0x7f7c1c101350 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.709+0000 7f7c137fe640 1 -- 172.21.15.46:0/4197667989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c0c0043c0 con 0x7f7c1c101350 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.709+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c1a6a80 con 0x7f7c1c101350 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.709+0000 7f7c137fe640 1 -- 172.21.15.46:0/4197667989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c004a00 con 0x7f7c1c101350 2024-09-12T02:57:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.709+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c1a6fd0 con 0x7f7c1c101350 2024-09-12T02:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 Processor -- start 2024-09-12T02:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 -- start start 2024-09-12T02:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c137fe640 1 -- 172.21.15.46:0/4197667989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c0c004ba0 con 0x7f7c1c101350 2024-09-12T02:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 0x55bb9a69ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a6012f0 con 0x55bb9a69d410 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a5a9270 con 0x55bb9a69caf0 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c2704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a5fd2b0 con 0x55bb9a6a7cb0 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c22048640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c22048640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59254/0 (socket says 172.21.15.46:59254) 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c22048640 1 -- 172.21.15.46:0/2572051292 learned_addr learned my addr 172.21.15.46:0/2572051292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.712+0000 7f7c22849640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 0x55bb9a69ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c21847640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 -- 172.21.15.46:0/2572051292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 msgr2=0x55bb9a6aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 -- 172.21.15.46:0/2572051292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 msgr2=0x55bb9a6a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 -- 172.21.15.46:0/2572051292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9a5a8340 con 0x55bb9a69caf0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c21847640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c137fe640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 0x7f7c080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22849640 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 0x55bb9a69ced0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c14005e30 tx=0x7f7c14009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a0caa8b63286c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c22048640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 0x7f7c080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c10bfe640 1 -- 172.21.15.46:0/2572051292 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14031850 con 0x55bb9a69caf0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c10bfe640 1 -- 172.21.15.46:0/2572051292 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c14036040 con 0x55bb9a69caf0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c10bfe640 1 -- 172.21.15.46:0/2572051292 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14004440 con 0x55bb9a69caf0 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.713+0000 7f7c137fe640 1 -- 172.21.15.46:0/4197667989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f7c0c0cf570 con 0x7f7c1c101350 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 -- 172.21.15.46:0/2572051292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 msgr2=0x55bb9a69ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 0x55bb9a69ced0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c14005e30 tx=0x7f7c14009a20 comp rx=0 tx=0).stop 2024-09-12T02:57:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c22048640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 0x7f7c080a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55bb9a699ec0 tx=0x7f7c1800b5e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 -- 172.21.15.46:0/2572051292 shutdown_connections 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 --2- 172.21.15.46:0/2572051292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a69caf0 0x55bb9a69ced0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 -- 172.21.15.46:0/2572051292 >> 172.21.15.46:0/2572051292 conn(0x55bb9a699060 msgr2=0x55bb9a699440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 -- 172.21.15.46:0/2572051292 shutdown_connections 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.714+0000 7f7c2704b740 1 -- 172.21.15.46:0/2572051292 wait complete. 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 -- start start 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 Processor -- start 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a6aa880 con 0x55bb9a69d410 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a58a290 con 0x55bb9a6a7cb0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb9a6aaa00 con 0x55bb9a69caf0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c2704b740 10 client.0 _ll_register_callbacks cb 0x55bb9a5bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59270/0 (socket says 172.21.15.46:59270) 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c21847640 1 -- 172.21.15.46:0/3517059101 learned_addr learned my addr 172.21.15.46:0/3517059101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c22048640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.715+0000 7f7c22849640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 -- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 msgr2=0x55bb9a6a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 -- 172.21.15.46:0/3517059101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 msgr2=0x55bb9a6a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 -- 172.21.15.46:0/3517059101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9a314210 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c21847640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005fd0 tx=0x7f7c0c0d2e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6403256c85db9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c01d040 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c14036040 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f7c0c0050b0 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c0c0b4070 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c2704b740 10 client.15921 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb9a3e0620 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.716+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c14036040 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7c0c097d50 con 0x55bb9a69d410 2024-09-12T02:57:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 10 client.15921 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 20 client.15921 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a6b7c80 0x55bb9a6ba070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 4 client.15921 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb9a60ea20 con 0x55bb9a6b7c80 2024-09-12T02:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c22048640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a6b7c80 0x55bb9a6ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c22048640 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a6b7c80 0x55bb9a6ba070 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33e942555e8c864 server_cookie=bc9f3f82838af1e4 in_seq=0 out_seq=0 2024-09-12T02:57:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.717+0000 7f7c05ffb640 10 client.15921 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c05ffb640 1 -- 172.21.15.46:0/3517059101 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb9a60ea20 con 0x55bb9a6b7c80 2024-09-12T02:57:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c05ffb640 10 client.15921 handle_command_reply: tid=0 2024-09-12T02:57:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 client.15921 shutdown 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 msgr2=0x55bb9a6a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005fd0 tx=0x7f7c0c0d2e50 comp rx=0 tx=0).stop 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 shutdown_connections 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a6b7c80 0x55bb9a6ba070 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6a50f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 --2- 172.21.15.46:0/3517059101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 >> 172.21.15.46:0/3517059101 conn(0x55bb9a699060 msgr2=0x55bb9a69edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c05ffb640 0 client.15921 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 shutdown_connections 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.718+0000 7f7c2704b740 1 -- 172.21.15.46:0/3517059101 wait complete. 2024-09-12T02:57:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.719+0000 7f7c2704b740 20 client.15921 trim_cache size 0 max 16384 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.734+0000 7f7c2704b740 1 Processor -- start 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.734+0000 7f7c2704b740 1 -- start start 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 0x55bb9a6bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1b3980 con 0x55bb9a69caf0 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1b11f0 con 0x55bb9a6a7cb0 2024-09-12T02:57:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c2704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1af060 con 0x55bb9a69d410 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59282/0 (socket says 172.21.15.46:59282) 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 -- 172.21.15.46:0/4082026381 learned_addr learned my addr 172.21.15.46:0/4082026381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22849640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c21847640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 0x55bb9a6bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 -- 172.21.15.46:0/4082026381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 msgr2=0x55bb9a6bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 0x55bb9a6bf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 -- 172.21.15.46:0/4082026381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 msgr2=0x55bb9a6bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 -- 172.21.15.46:0/4082026381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9a69f940 con 0x55bb9a69caf0 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c21847640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 0x55bb9a6bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22849640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c22048640 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800bfd0 tx=0x7f7c18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1abe296d0bff1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.735+0000 7f7c07fff640 1 -- 172.21.15.46:0/4082026381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1802e070 con 0x55bb9a69caf0 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c07fff640 1 -- 172.21.15.46:0/4082026381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c18028840 con 0x55bb9a69caf0 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c07fff640 1 -- 172.21.15.46:0/4082026381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1801fa60 con 0x55bb9a69caf0 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 -- 172.21.15.46:0/4082026381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 msgr2=0x55bb9a6a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800bfd0 tx=0x7f7c18015000 comp rx=0 tx=0).stop 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 -- 172.21.15.46:0/4082026381 shutdown_connections 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69d410 0x55bb9a6bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 --2- 172.21.15.46:0/4082026381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69caf0 0x55bb9a6a5460 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 -- 172.21.15.46:0/4082026381 >> 172.21.15.46:0/4082026381 conn(0x55bb9a699060 msgr2=0x55bb9a6a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 -- 172.21.15.46:0/4082026381 shutdown_connections 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.736+0000 7f7c2704b740 1 -- 172.21.15.46:0/4082026381 wait complete. 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 -- start start 2024-09-12T02:57:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 Processor -- start 2024-09-12T02:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1af6b0 con 0x55bb9a69d410 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1b0b20 con 0x55bb9a6a7cb0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1afd80 con 0x55bb9a69caf0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c2704b740 10 client.0 _ll_register_callbacks cb 0x7f7c0c004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c21847640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59298/0 (socket says 172.21.15.46:59298) 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.737+0000 7f7c21847640 1 -- 172.21.15.46:0/1052375716 learned_addr learned my addr 172.21.15.46:0/1052375716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c22849640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c22048640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 -- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 msgr2=0x55bb9a6abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 -- 172.21.15.46:0/1052375716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 msgr2=0x55bb9a6ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 -- 172.21.15.46:0/1052375716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9a721ae0 con 0x55bb9a69d410 2024-09-12T02:57:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c22849640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c21847640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0cc9c0 tx=0x7f7c0c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b729695da2e61698 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c01d070 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f7c0c09ad40 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c18028840 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c0c09b040 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c2704b740 10 client.15927 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb9a722d60 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.738+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c18028840 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7c0c060be0 con 0x55bb9a69d410 2024-09-12T02:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 20 client.15927 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a723290 0x55bb9a725680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 4 client.15927 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb9a608d70 con 0x55bb9a723290 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c22048640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a723290 0x55bb9a725680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c22048640 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a723290 0x55bb9a725680 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef35cc69ad7b61 server_cookie=956d10aa01117c4 in_seq=0 out_seq=0 2024-09-12T02:57:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.739+0000 7f7c05ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.762+0000 7f7c05ffb640 1 -- 172.21.15.46:0/1052375716 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb9a608d70 con 0x55bb9a723290 2024-09-12T02:57:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.762+0000 7f7c05ffb640 10 client.15927 handle_command_reply: tid=0 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.762+0000 7f7c2704b740 1 client.15927 shutdown 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.762+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 msgr2=0x55bb9a6ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.762+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0cc9c0 tx=0x7f7c0c0052e0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 shutdown_connections 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55bb9a723290 0x55bb9a725680 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bb9a6a7cb0 0x55bb9a6ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c05ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb9a69d410 0x55bb9a6ac220 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 --2- 172.21.15.46:0/1052375716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb9a69caf0 0x55bb9a6abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 >> 172.21.15.46:0/1052375716 conn(0x55bb9a699060 msgr2=0x55bb9a6979a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 shutdown_connections 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 1 -- 172.21.15.46:0/1052375716 wait complete. 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c2704b740 20 client.15927 trim_cache size 0 max 16384 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:57:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 msgr2=0x7f7c080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 0x7f7c080a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55bb9a699ec0 tx=0x7f7c1800b5e0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 msgr2=0x7f7c1c102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.763+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c890 tx=0x7f7c0c00cd50 comp rx=0 tx=0).stop 2024-09-12T02:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 shutdown_connections 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f7c080a2da0 0x7f7c080a5260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c1c1069c0 0x7f7c1c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c1c101350 0x7f7c1c102ac0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 --2- 172.21.15.46:0/4197667989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c1c1009b0 0x7f7c1c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 >> 172.21.15.46:0/4197667989 conn(0x7f7c1c077990 msgr2=0x7f7c1c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 shutdown_connections 2024-09-12T02:57:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:26.764+0000 7f7c242ac640 1 -- 172.21.15.46:0/4197667989 wait complete. 2024-09-12T02:57:27.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:27 smithi196 ceph-mon[34587]: pgmap v223: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:28.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:27 smithi046 ceph-mon[30951]: pgmap v223: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:28.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:27 smithi145 ceph-mon[34674]: pgmap v223: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:29.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:29 smithi196 ceph-mon[34587]: pgmap v224: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:30.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:29 smithi046 ceph-mon[30951]: pgmap v224: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:29 smithi145 ceph-mon[34674]: pgmap v224: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:31.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:31 smithi196 ceph-mon[34587]: pgmap v225: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:32.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:31 smithi046 ceph-mon[30951]: pgmap v225: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:32.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:31 smithi145 ceph-mon[34674]: pgmap v225: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:57:33.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:33 smithi196 ceph-mon[34587]: pgmap v226: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 33 op/s 2024-09-12T02:57:34.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:33 smithi046 ceph-mon[30951]: pgmap v226: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 33 op/s 2024-09-12T02:57:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:33 smithi145 ceph-mon[34674]: pgmap v226: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 33 op/s 2024-09-12T02:57:35.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:35 smithi196 ceph-mon[34587]: pgmap v227: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:36.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:35 smithi046 ceph-mon[30951]: pgmap v227: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:36.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:35 smithi145 ceph-mon[34674]: pgmap v227: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:37.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:37 smithi196 ceph-mon[34587]: pgmap v228: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-09-12T02:57:38.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:37 smithi046 ceph-mon[30951]: pgmap v228: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-09-12T02:57:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:37 smithi145 ceph-mon[34674]: pgmap v228: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.870+0000 7f43e213b640 1 -- 172.21.15.46:0/800773663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 msgr2=0x7f43dc1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.870+0000 7f43e213b640 1 --2- 172.21.15.46:0/800773663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1030c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009900 tx=0x7f43cc02a480 comp rx=0 tx=0).stop 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 -- 172.21.15.46:0/800773663 shutdown_connections 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 --2- 172.21.15.46:0/800773663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 0x7f43dc10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 --2- 172.21.15.46:0/800773663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 0x7f43dc10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 --2- 172.21.15.46:0/800773663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1030c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 -- 172.21.15.46:0/800773663 >> 172.21.15.46:0/800773663 conn(0x7f43dc077990 msgr2=0x7f43dc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 -- 172.21.15.46:0/800773663 shutdown_connections 2024-09-12T02:57:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 -- 172.21.15.46:0/800773663 wait complete. 2024-09-12T02:57:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 Processor -- start 2024-09-12T02:57:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.871+0000 7f43e213b640 1 -- start start 2024-09-12T02:57:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 0x7f43dc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 0x7f43dc19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc111d60 con 0x7f43dc102ce0 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc111be0 con 0x7f43dc10c0b0 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43e213b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc111ee0 con 0x7f43dc103600 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38078/0 (socket says 172.21.15.46:38078) 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 -- 172.21.15.46:0/2320052720 learned_addr learned my addr 172.21.15.46:0/2320052720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43daffd640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 0x7f43dc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43dbfff640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 0x7f43dc19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 -- 172.21.15.46:0/2320052720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 msgr2=0x7f43dc1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 0x7f43dc1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 -- 172.21.15.46:0/2320052720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 msgr2=0x7f43dc19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 0x7f43dc19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 -- 172.21.15.46:0/2320052720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc19d290 con 0x7f43dc102ce0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.872+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005940 tx=0x7f43cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.873+0000 7f43d8ff9640 1 -- 172.21.15.46:0/2320052720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc03f030 con 0x7f43dc102ce0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.873+0000 7f43d8ff9640 1 -- 172.21.15.46:0/2320052720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc003cb0 con 0x7f43dc102ce0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.873+0000 7f43d8ff9640 1 -- 172.21.15.46:0/2320052720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc035070 con 0x7f43dc102ce0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.873+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc19d520 con 0x7f43dc102ce0 2024-09-12T02:57:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.873+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc1a4e10 con 0x7f43dc102ce0 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 Processor -- start 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 -- start start 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43d8ff9640 1 -- 172.21.15.46:0/2320052720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f43cc042030 con 0x7f43dc102ce0 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec7af0 0x564aa5ec7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564aa5e2c2f0 con 0x564aa5ec8410 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564aa5dd4270 con 0x564aa5ec7af0 2024-09-12T02:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564aa5e282b0 con 0x564aa5ed2cb0 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38086/0 (socket says 172.21.15.46:38086) 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 -- 172.21.15.46:0/2385179014 learned_addr learned my addr 172.21.15.46:0/2385179014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43daffd640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43dbfff640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec7af0 0x564aa5ec7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 -- 172.21.15.46:0/2385179014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 msgr2=0x564aa5ed50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 -- 172.21.15.46:0/2385179014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec7af0 msgr2=0x564aa5ec7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.876+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec7af0 0x564aa5ec7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43db7fe640 1 -- 172.21.15.46:0/2385179014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa5dd3340 con 0x564aa5ec8410 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43daffd640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43d8ff9640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 0x7f43c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43db7fe640 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ed2e0 tx=0x7f43cc0f8320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e769de68916d8337 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43dbfff640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 0x7f43c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43d8ff9640 1 -- 172.21.15.46:0/2320052720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f43cc0effd0 con 0x7f43dc102ce0 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43c1ffb640 1 -- 172.21.15.46:0/2385179014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b8490 con 0x564aa5ec8410 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43c1ffb640 1 -- 172.21.15.46:0/2385179014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc0bc1f0 con 0x564aa5ec8410 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.877+0000 7f43c1ffb640 1 -- 172.21.15.46:0/2385179014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0bbb50 con 0x564aa5ec8410 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- 172.21.15.46:0/2385179014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 msgr2=0x564aa5ed2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ed2e0 tx=0x7f43cc0f8320 comp rx=0 tx=0).stop 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43dbfff640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 0x7f43c40a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4009600 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- 172.21.15.46:0/2385179014 shutdown_connections 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5ed2770 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/2385179014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec7af0 0x564aa5ec7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- 172.21.15.46:0/2385179014 >> 172.21.15.46:0/2385179014 conn(0x564aa5ec4060 msgr2=0x564aa5ec4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- 172.21.15.46:0/2385179014 shutdown_connections 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- 172.21.15.46:0/2385179014 wait complete. 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 -- start start 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.878+0000 7f43e4ed7740 1 Processor -- start 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec8410 0x564aa5ed0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564aa5ed5640 con 0x564aa5ec7af0 2024-09-12T02:57:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564aa5db5290 con 0x564aa5ec8410 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564aa5ed57c0 con 0x564aa5ed2cb0 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43e4ed7740 10 client.0 _ll_register_callbacks cb 0x564aa5de6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38102/0 (socket says 172.21.15.46:38102) 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 -- 172.21.15.46:0/213643920 learned_addr learned my addr 172.21.15.46:0/213643920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43daffd640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec8410 0x564aa5ed0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 -- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 msgr2=0x564aa5ed0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 -- 172.21.15.46:0/213643920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec8410 msgr2=0x564aa5ed0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec8410 0x564aa5ed0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 -- 172.21.15.46:0/213643920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa5b3f210 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43db7fe640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ed420 tx=0x7f43cc0f36e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5bc608f2e14d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43baffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.879+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc049050 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f43cc0dc110 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0bc1f0 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43cc0bbd30 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 10 client.15939 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564aa5c0b620 con 0x564aa5ec7af0 2024-09-12T02:57:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc0bc1f0 con 0x564aa5ec7af0 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f43cc0c0160 con 0x564aa5ec7af0 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 10 client.15939 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 20 client.15939 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5ee4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 4 client.15939 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.880+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa5e39a20 con 0x564aa5ee2800 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.881+0000 7f43daffd640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5ee4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.881+0000 7f43baffd640 10 client.15939 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.881+0000 7f43daffd640 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5ee4bf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbdd425c46edf641 server_cookie=d4f619cffe5c510f in_seq=0 out_seq=0 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43baffd640 1 -- 172.21.15.46:0/213643920 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564aa5e39a20 con 0x564aa5ee2800 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43baffd640 10 client.15939 handle_command_reply: tid=0 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 client.15939 shutdown 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 msgr2=0x564aa5ecfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ed420 tx=0x7f43cc0f36e0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 shutdown_connections 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5ee4bf0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43baffd640 0 client.15939 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ec8410 0x564aa5ed0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/213643920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ecfc00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 >> 172.21.15.46:0/213643920 conn(0x564aa5ec4060 msgr2=0x564aa5ec9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 shutdown_connections 2024-09-12T02:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 1 -- 172.21.15.46:0/213643920 wait complete. 2024-09-12T02:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.882+0000 7f43e4ed7740 20 client.15939 trim_cache size 0 max 16384 2024-09-12T02:57:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 Processor -- start 2024-09-12T02:57:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 -- start start 2024-09-12T02:57:39.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec8410 0x564aa5ed1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5eea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1b17f0 con 0x564aa5ec7af0 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1af060 con 0x564aa5ed2cb0 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1aced0 con 0x564aa5ec8410 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38114/0 (socket says 172.21.15.46:38114) 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 -- 172.21.15.46:0/3211345465 learned_addr learned my addr 172.21.15.46:0/3211345465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43daffd640 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec8410 0x564aa5ed1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43dbfff640 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5eea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 -- 172.21.15.46:0/3211345465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec8410 msgr2=0x564aa5ed1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec8410 0x564aa5ed1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 -- 172.21.15.46:0/3211345465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 msgr2=0x564aa5eea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5eea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 -- 172.21.15.46:0/3211345465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa5ec7870 con 0x564aa5ec7af0 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.898+0000 7f43db7fe640 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ba4a0 tx=0x7f43cc0f8320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7591c15b12153625 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.899+0000 7f43c17fa640 1 -- 172.21.15.46:0/3211345465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc049070 con 0x564aa5ec7af0 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.899+0000 7f43c17fa640 1 -- 172.21.15.46:0/3211345465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc0bc1f0 con 0x564aa5ec7af0 2024-09-12T02:57:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.899+0000 7f43c17fa640 1 -- 172.21.15.46:0/3211345465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0bbfc0 con 0x564aa5ec7af0 2024-09-12T02:57:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- 172.21.15.46:0/3211345465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 msgr2=0x564aa5ed04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ba4a0 tx=0x7f43cc0f8320 comp rx=0 tx=0).stop 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- 172.21.15.46:0/3211345465 shutdown_connections 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5eea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec8410 0x564aa5ed1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/3211345465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec7af0 0x564aa5ed04b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- 172.21.15.46:0/3211345465 >> 172.21.15.46:0/3211345465 conn(0x564aa5ec4060 msgr2=0x564aa5ed1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- 172.21.15.46:0/3211345465 shutdown_connections 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- 172.21.15.46:0/3211345465 wait complete. 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 -- start start 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 Processor -- start 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec7af0 0x564aa5edacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.900+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1ad520 con 0x564aa5ec8410 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1ae990 con 0x564aa5ed2cb0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1adbf0 con 0x564aa5ec7af0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 10 client.0 _ll_register_callbacks cb 0x7f43d4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38128/0 (socket says 172.21.15.46:38128) 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 -- 172.21.15.46:0/1940698907 learned_addr learned my addr 172.21.15.46:0/1940698907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43db7fe640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec7af0 0x564aa5edacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43dbfff640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 -- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec7af0 msgr2=0x564aa5edacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec7af0 0x564aa5edacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 -- 172.21.15.46:0/1940698907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 msgr2=0x564aa5ed6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 -- 172.21.15.46:0/1940698907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa5ed68a0 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43daffd640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004700 tx=0x7f43d000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d00fd85a42c559 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43baffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0005020 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f43d00051c0 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.901+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0bc1f0 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43d002e050 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 10 client.15945 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564aa5f4d030 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43cc0bc1f0 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f43d001d380 con 0x564aa5ec8410 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 10 client.15945 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 20 client.15945 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5f4f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 4 client.15945 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aa5e33d70 con 0x564aa5ee2800 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.902+0000 7f43db7fe640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5f4f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.903+0000 7f43baffd640 10 client.15945 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.903+0000 7f43db7fe640 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5f4f820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2826c796f358a322 server_cookie=16242c59e9273861 in_seq=0 out_seq=0 2024-09-12T02:57:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43baffd640 1 -- 172.21.15.46:0/1940698907 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564aa5e33d70 con 0x564aa5ee2800 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43baffd640 10 client.15945 handle_command_reply: tid=0 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 client.15945 shutdown 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 msgr2=0x564aa5edb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004700 tx=0x7f43d000bd80 comp rx=0 tx=0).stop 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 shutdown_connections 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564aa5ee2800 0x564aa5f4f820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564aa5ed2cb0 0x564aa5ed6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43baffd640 0 client.15945 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564aa5ec8410 0x564aa5edb230 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 --2- 172.21.15.46:0/1940698907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564aa5ec7af0 0x564aa5edacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 >> 172.21.15.46:0/1940698907 conn(0x564aa5ec4060 msgr2=0x564aa5ece070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 shutdown_connections 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 1 -- 172.21.15.46:0/1940698907 wait complete. 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.912+0000 7f43e4ed7740 20 client.15945 trim_cache size 0 max 16384 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:57:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 msgr2=0x7f43c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 0x7f43c40a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005fd0 tx=0x7f43d4009600 comp rx=0 tx=0).stop 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 msgr2=0x7f43dc1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005940 tx=0x7f43cc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 shutdown_connections 2024-09-12T02:57:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f43c40a2da0 0x7f43c40a5260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc10c0b0 0x7f43dc19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43dc103600 0x7f43dc1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 --2- 172.21.15.46:0/2320052720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43dc102ce0 0x7f43dc1982a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 >> 172.21.15.46:0/2320052720 conn(0x7f43dc077990 msgr2=0x7f43dc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 shutdown_connections 2024-09-12T02:57:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:39.913+0000 7f43e213b640 1 -- 172.21.15.46:0/2320052720 wait complete. 2024-09-12T02:57:39.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:39 smithi196 ceph-mon[34587]: pgmap v229: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:40.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:39 smithi046 ceph-mon[30951]: pgmap v229: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:39 smithi145 ceph-mon[34674]: pgmap v229: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:41.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:41 smithi196 ceph-mon[34587]: pgmap v230: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:42.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:41 smithi046 ceph-mon[30951]: pgmap v230: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:41 smithi145 ceph-mon[34674]: pgmap v230: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-12T02:57:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:43 smithi196 ceph-mon[34587]: pgmap v231: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-12T02:57:44.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:43 smithi046 ceph-mon[30951]: pgmap v231: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-12T02:57:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:43 smithi145 ceph-mon[34674]: pgmap v231: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-12T02:57:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:45 smithi196 ceph-mon[34587]: pgmap v232: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:45 smithi046 ceph-mon[30951]: pgmap v232: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:45 smithi145 ceph-mon[34674]: pgmap v232: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:57:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:47 smithi196 ceph-mon[34587]: pgmap v233: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:57:48.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:47 smithi046 ceph-mon[30951]: pgmap v233: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:57:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:47 smithi145 ceph-mon[34674]: pgmap v233: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 24 op/s 2024-09-12T02:57:49.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:49 smithi196 ceph-mon[34587]: pgmap v234: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:50.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:49 smithi046 ceph-mon[30951]: pgmap v234: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:50.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:49 smithi145 ceph-mon[34674]: pgmap v234: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:51 smithi196 ceph-mon[34587]: pgmap v235: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:52.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:51 smithi046 ceph-mon[30951]: pgmap v235: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:52.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:51 smithi145 ceph-mon[34674]: pgmap v235: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:57:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 -- 172.21.15.46:0/3534171358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 msgr2=0x7f298c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 --2- 172.21.15.46:0/3534171358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c100e10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2980009930 tx=0x7f298002a420 comp rx=0 tx=0).stop 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 -- 172.21.15.46:0/3534171358 shutdown_connections 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 --2- 172.21.15.46:0/3534171358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 0x7f298c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 --2- 172.21.15.46:0/3534171358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c100e10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 --2- 172.21.15.46:0/3534171358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 0x7f298c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 -- 172.21.15.46:0/3534171358 >> 172.21.15.46:0/3534171358 conn(0x7f298c077990 msgr2=0x7f298c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 -- 172.21.15.46:0/3534171358 shutdown_connections 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 -- 172.21.15.46:0/3534171358 wait complete. 2024-09-12T02:57:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.009+0000 7f29917f1640 1 Processor -- start 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 -- start start 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 0x7f298c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 0x7f298c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f298c110d00 con 0x7f298c1009b0 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f298c110b80 con 0x7f298c1069c0 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f29917f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f298c110e80 con 0x7f298c101350 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40040/0 (socket says 172.21.15.46:40040) 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 -- 172.21.15.46:0/2930909019 learned_addr learned my addr 172.21.15.46:0/2930909019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298a7fc640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 0x7f298c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298b7fe640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 0x7f298c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 -- 172.21.15.46:0/2930909019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 msgr2=0x7f298c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 0x7f298c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 -- 172.21.15.46:0/2930909019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 msgr2=0x7f298c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 0x7f298c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.010+0000 7f298affd640 1 -- 172.21.15.46:0/2930909019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c19f460 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f298affd640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f297c00d950 tx=0x7f297c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f2973fff640 1 -- 172.21.15.46:0/2930909019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c017070 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c19f750 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f2973fff640 1 -- 172.21.15.46:0/2930909019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f297c009c30 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f2973fff640 1 -- 172.21.15.46:0/2930909019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c00f620 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.011+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1a6fe0 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.013+0000 7f2973fff640 1 -- 172.21.15.46:0/2930909019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f297c0109a0 con 0x7f298c1009b0 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.013+0000 7f2994664740 1 Processor -- start 2024-09-12T02:57:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 -- start start 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ebeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3e232f0 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3dcb270 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2994664740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3e1f2b0 con 0x55d9d3ebf410 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f298affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40052/0 (socket says 172.21.15.46:40052) 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f298a7fc640 1 -- 172.21.15.46:0/209422150 learned_addr learned my addr 172.21.15.46:0/209422150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f2973fff640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 0x7f29740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.014+0000 7f298b7fe640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ebeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f2973fff640 1 -- 172.21.15.46:0/2930909019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f297c0cf480 con 0x7f298c1009b0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 -- 172.21.15.46:0/209422150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 msgr2=0x55d9d3ec9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 -- 172.21.15.46:0/209422150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 msgr2=0x55d9d3ebeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ebeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 -- 172.21.15.46:0/209422150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d3dca340 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298b7fe640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ebeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298affd640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298b7fe640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 0x7f29740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f298a7fc640 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2980031440 tx=0x7f2980004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a70c8db1b632132 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f29717fa640 1 -- 172.21.15.46:0/209422150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980035030 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f29717fa640 1 -- 172.21.15.46:0/209422150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29800054f0 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.015+0000 7f29717fa640 1 -- 172.21.15.46:0/209422150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800035f0 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- 172.21.15.46:0/209422150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 msgr2=0x55d9d3ecc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f298b7fe640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 0x7f29740a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2984005fd0 tx=0x7f2984004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2980031440 tx=0x7f2980004160 comp rx=0 tx=0).stop 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- 172.21.15.46:0/209422150 shutdown_connections 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecc0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 --2- 172.21.15.46:0/209422150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ebeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- 172.21.15.46:0/209422150 >> 172.21.15.46:0/209422150 conn(0x55d9d3ebb060 msgr2=0x55d9d3ebb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- 172.21.15.46:0/209422150 shutdown_connections 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- 172.21.15.46:0/209422150 wait complete. 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 -- start start 2024-09-12T02:57:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.016+0000 7f2994664740 1 Processor -- start 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ec7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3ecc880 con 0x55d9d3ebf410 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3dac290 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9d3ecca00 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f2994664740 10 client.0 _ll_register_callbacks cb 0x55d9d3ddd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40062/0 (socket says 172.21.15.46:40062) 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 -- 172.21.15.46:0/2968773370 learned_addr learned my addr 172.21.15.46:0/2968773370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298affd640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 -- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebeaf0 msgr2=0x55d9d3ec6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 -- 172.21.15.46:0/2968773370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ec9cb0 msgr2=0x55d9d3ec7600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ec7600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 -- 172.21.15.46:0/2968773370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d3b36210 con 0x55d9d3ebf410 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f298a7fc640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2980000c00 tx=0x7f2980004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f7e5aa4c63eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.017+0000 7f296eff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980035070 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f2980005180 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29800054f0 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2980057050 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 10 client.15957 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9d3c02620 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29800054f0 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2980031660 con 0x55d9d3ebf410 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 20 client.15957 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3edbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 4 client.15957 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.018+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9d3e30a20 con 0x55d9d3ed9ab0 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.019+0000 7f298affd640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3edbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.019+0000 7f298affd640 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3edbea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e134a8790b72571e server_cookie=7e2cd97d6c43e08 in_seq=0 out_seq=0 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.019+0000 7f296eff5640 10 client.15957 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f296eff5640 1 -- 172.21.15.46:0/2968773370 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9d3e30a20 con 0x55d9d3ed9ab0 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f296eff5640 10 client.15957 handle_command_reply: tid=0 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 client.15957 shutdown 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 msgr2=0x55d9d3ec70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2980000c00 tx=0x7f2980004ba0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 shutdown_connections 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3edbea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ec7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f296eff5640 0 client.15957 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec70c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 --2- 172.21.15.46:0/2968773370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 >> 172.21.15.46:0/2968773370 conn(0x55d9d3ebb060 msgr2=0x55d9d3ec0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 shutdown_connections 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 1 -- 172.21.15.46:0/2968773370 wait complete. 2024-09-12T02:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.020+0000 7f2994664740 20 client.15957 trim_cache size 0 max 16384 2024-09-12T02:57:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.037+0000 7f2994664740 1 Processor -- start 2024-09-12T02:57:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.037+0000 7f2994664740 1 -- start start 2024-09-12T02:57:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.037+0000 7f2994664740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.037+0000 7f2994664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.037+0000 7f2994664740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ee1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2994664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1b38a0 con 0x55d9d3ebf410 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2994664740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1b1110 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2994664740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1aef80 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40064/0 (socket says 172.21.15.46:40064) 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 -- 172.21.15.46:0/3737616488 learned_addr learned my addr 172.21.15.46:0/3737616488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298affd640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298b7fe640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ee1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 -- 172.21.15.46:0/3737616488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 msgr2=0x55d9d3ee1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ee1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 -- 172.21.15.46:0/3737616488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 msgr2=0x55d9d3ec7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 -- 172.21.15.46:0/3737616488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d3ebf240 con 0x55d9d3ebf410 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298affd640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2980031ba0 tx=0x7f29800058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44712dfb3340085b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2970ff9640 1 -- 172.21.15.46:0/3737616488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980035070 con 0x55d9d3ebf410 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2970ff9640 1 -- 172.21.15.46:0/3737616488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2980004f80 con 0x55d9d3ebf410 2024-09-12T02:57:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.038+0000 7f2970ff9640 1 -- 172.21.15.46:0/3737616488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980048070 con 0x55d9d3ebf410 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 -- 172.21.15.46:0/3737616488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 msgr2=0x55d9d3ec8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2980031ba0 tx=0x7f29800058b0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 -- 172.21.15.46:0/3737616488 shutdown_connections 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ee1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebf410 0x55d9d3ec8af0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 --2- 172.21.15.46:0/3737616488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3ec7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.039+0000 7f2994664740 1 -- 172.21.15.46:0/3737616488 >> 172.21.15.46:0/3737616488 conn(0x55d9d3ebb060 msgr2=0x55d9d3ec83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- 172.21.15.46:0/3737616488 shutdown_connections 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- 172.21.15.46:0/3737616488 wait complete. 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- start start 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 Processor -- start 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 0x55d9d3f3cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1af5d0 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1b0a40 con 0x55d9d3ebf410 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1afca0 con 0x55d9d3ec9cb0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f2994664740 10 client.0 _ll_register_callbacks cb 0x7f297c097910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f298affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f298affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40070/0 (socket says 172.21.15.46:40070) 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.040+0000 7f298affd640 1 -- 172.21.15.46:0/3461455803 learned_addr learned my addr 172.21.15.46:0/3461455803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 0x55d9d3f3cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298b7fe640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 -- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 msgr2=0x55d9d3ecd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 -- 172.21.15.46:0/3461455803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 msgr2=0x55d9d3f3cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 0x55d9d3f3cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 -- 172.21.15.46:0/3461455803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d3ecd9e0 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298b7fe640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 0x55d9d3f3cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f298affd640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c0979c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e5f013005269b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f296eff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c020040 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9467+0+0 (secure 0 0 0) 0x7f297c097c60 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2980004f80 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f297c009de0 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f2994664740 10 client.15963 fetch_fsmap learned FSMap version 31 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9d3f44050 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.041+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2980004f80 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f297c0cf120 con 0x55d9d3ebeaf0 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 20 client.15963 populate_metadata read hostname 'smithi046' 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3f467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 4 client.15963 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9d3e2ad70 con 0x55d9d3ed9ab0 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3f467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f298a7fc640 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3f467b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8268102af25cbd17 server_cookie=ceffd0ebbdde4b64 in_seq=0 out_seq=0 2024-09-12T02:57:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.042+0000 7f296eff5640 10 client.15963 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f296eff5640 1 -- 172.21.15.46:0/3461455803 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9d3e2ad70 con 0x55d9d3ed9ab0 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f296eff5640 10 client.15963 handle_command_reply: tid=0 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 client.15963 shutdown 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 msgr2=0x55d9d3f3c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c0979c0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 shutdown_connections 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55d9d3ed9ab0 0x55d9d3f467b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9d3ec9cb0 0x55d9d3ecd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d9d3ebf410 0x55d9d3f3cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f296eff5640 0 client.15963 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 --2- 172.21.15.46:0/3461455803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9d3ebeaf0 0x55d9d3f3c800 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 >> 172.21.15.46:0/3461455803 conn(0x55d9d3ebb060 msgr2=0x55d9d3eb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 shutdown_connections 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 1 -- 172.21.15.46:0/3461455803 wait complete. 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.052+0000 7f2994664740 20 client.15963 trim_cache size 0 max 16384 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:57:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:57:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 msgr2=0x7f29740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 0x7f29740a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2984005fd0 tx=0x7f2984004a40 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 msgr2=0x7f298c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f297c00d950 tx=0x7f297c00de10 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 shutdown_connections 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29740a2da0 0x7f29740a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f298c1069c0 0x7f298c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f298c101350 0x7f298c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 --2- 172.21.15.46:0/2930909019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298c1009b0 0x7f298c19a470 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 >> 172.21.15.46:0/2930909019 conn(0x7f298c077990 msgr2=0x7f298c069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 shutdown_connections 2024-09-12T02:57:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:57:53.053+0000 7f29917f1640 1 -- 172.21.15.46:0/2930909019 wait complete. 2024-09-12T02:57:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:53 smithi196 ceph-mon[34587]: pgmap v236: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-09-12T02:57:54.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:53 smithi046 ceph-mon[30951]: pgmap v236: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-09-12T02:57:54.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:53 smithi145 ceph-mon[34674]: pgmap v236: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-09-12T02:57:55.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:55 smithi196 ceph-mon[34587]: pgmap v237: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:57:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:55 smithi046 ceph-mon[30951]: pgmap v237: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:57:56.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:55 smithi145 ceph-mon[34674]: pgmap v237: 129 pgs: 129 active+clean; 9.9 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 14 op/s 2024-09-12T02:57:57.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:57 smithi196 ceph-mon[34587]: pgmap v238: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:57 smithi046 ceph-mon[30951]: pgmap v238: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:57 smithi145 ceph-mon[34674]: pgmap v238: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:57:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:57:59 smithi196 ceph-mon[34587]: pgmap v239: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:57:59 smithi046 ceph-mon[30951]: pgmap v239: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:00.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:57:59 smithi145 ceph-mon[34674]: pgmap v239: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:02.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:02 smithi196 ceph-mon[34587]: pgmap v240: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:02 smithi046 ceph-mon[30951]: pgmap v240: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:02 smithi145 ceph-mon[34674]: pgmap v240: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:03.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:03 smithi046 ceph-mon[30951]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T02:58:03.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:03 smithi145 ceph-mon[34674]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T02:58:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:03 smithi196 ceph-mon[34587]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T02:58:04.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:04 smithi046 ceph-mon[30951]: pgmap v242: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:04 smithi145 ceph-mon[34674]: pgmap v242: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:04 smithi196 ceph-mon[34587]: pgmap v242: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:06.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:06 smithi046 ceph-mon[30951]: pgmap v243: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:06 smithi145 ceph-mon[34674]: pgmap v243: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 -- 172.21.15.46:0/337086980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101ad0 msgr2=0x7fae20101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 --2- 172.21.15.46:0/337086980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101ad0 0x7fae20101f30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae14009930 tx=0x7fae1402a420 comp rx=0 tx=0).stop 2024-09-12T02:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 -- 172.21.15.46:0/337086980 shutdown_connections 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 --2- 172.21.15.46:0/337086980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae2010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 --2- 172.21.15.46:0/337086980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20101ad0 0x7fae20101f30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 --2- 172.21.15.46:0/337086980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20107b70 0x7fae20107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 -- 172.21.15.46:0/337086980 >> 172.21.15.46:0/337086980 conn(0x7fae200fd7d0 msgr2=0x7fae200ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 -- 172.21.15.46:0/337086980 shutdown_connections 2024-09-12T02:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.678+0000 7fae251af640 1 -- 172.21.15.46:0/337086980 wait complete. 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.679+0000 7fae251af640 1 Processor -- start 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 -- start start 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 0x7fae20198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae201987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111de0 con 0x7fae20107b70 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111c60 con 0x7fae20101ad0 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae251af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111f60 con 0x7fae20102470 2024-09-12T02:58:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35980/0 (socket says 172.21.15.46:35980) 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 -- 172.21.15.46:0/1233938736 learned_addr learned my addr 172.21.15.46:0/1233938736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1e59c640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae201987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 0x7fae20198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 -- 172.21.15.46:0/1233938736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 msgr2=0x7fae201987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae201987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 -- 172.21.15.46:0/1233938736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 msgr2=0x7fae20198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 0x7fae20198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1f59e640 1 -- 172.21.15.46:0/1233938736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019d280 con 0x7fae20107b70 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1e59c640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae201987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.680+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 0x7fae20198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae1f59e640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae1800eb10 tx=0x7fae1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae07fff640 1 -- 172.21.15.46:0/1233938736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18004b20 con 0x7fae20107b70 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019d570 con 0x7fae20107b70 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae07fff640 1 -- 172.21.15.46:0/1233938736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18004cc0 con 0x7fae20107b70 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae07fff640 1 -- 172.21.15.46:0/1233938736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18010690 con 0x7fae20107b70 2024-09-12T02:58:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.681+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a4e00 con 0x7fae20107b70 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.683+0000 7fae07fff640 1 -- 172.21.15.46:0/1233938736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae18005020 con 0x7fae20107b70 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 Processor -- start 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 -- start start 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae07fff640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 0x7fae080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 0x56537cb56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56537cabb2f0 con 0x56537cb57410 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56537ca63270 con 0x56537cb56af0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae28024740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56537cab72b0 con 0x56537cb61cb0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.684+0000 7fae1e59c640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 0x7fae080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae07fff640 1 -- 172.21.15.46:0/1233938736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9555+0+0 (secure 0 0 0) 0x7fae180cffb0 con 0x7fae20107b70 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 0x56537cb56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35982/0 (socket says 172.21.15.46:35982) 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1e59c640 1 -- 172.21.15.46:0/463117060 learned_addr learned my addr 172.21.15.46:0/463117060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1f59e640 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 -- 172.21.15.46:0/463117060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 msgr2=0x56537cb640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 -- 172.21.15.46:0/463117060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 msgr2=0x56537cb61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 -- 172.21.15.46:0/463117060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537ca62340 con 0x56537cb56af0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1e59c640 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1e59c640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 0x7fae080a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae14004650 tx=0x7fae14005780 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 0x56537cb56ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae10009bb0 tx=0x7fae100113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14aeed02fdd962bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.685+0000 7fae057fa640 1 -- 172.21.15.46:0/463117060 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10011db0 con 0x56537cb56af0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae057fa640 1 -- 172.21.15.46:0/463117060 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10012900 con 0x56537cb56af0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae057fa640 1 -- 172.21.15.46:0/463117060 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000f840 con 0x56537cb56af0 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 -- 172.21.15.46:0/463117060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 msgr2=0x56537cb56ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 0x56537cb56ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae10009bb0 tx=0x7fae100113e0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 -- 172.21.15.46:0/463117060 shutdown_connections 2024-09-12T02:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb57410 0x56537cb61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 --2- 172.21.15.46:0/463117060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb56af0 0x56537cb56ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 -- 172.21.15.46:0/463117060 >> 172.21.15.46:0/463117060 conn(0x56537cb53060 msgr2=0x56537cb53440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 -- 172.21.15.46:0/463117060 shutdown_connections 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.686+0000 7fae28024740 1 -- 172.21.15.46:0/463117060 wait complete. 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 -- start start 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 Processor -- start 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 0x56537cb5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb5f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56537cb64880 con 0x56537cb61cb0 2024-09-12T02:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56537ca44290 con 0x56537cb57410 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56537cb64a00 con 0x56537cb56af0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae28024740 10 client.0 _ll_register_callbacks cb 0x56537ca75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae1f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae1f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35996/0 (socket says 172.21.15.46:35996) 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae1f59e640 1 -- 172.21.15.46:0/3917237573 learned_addr learned my addr 172.21.15.46:0/3917237573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.687+0000 7fae1e59c640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb5f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 0x56537cb5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 -- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 msgr2=0x56537cb5ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 0x56537cb5ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 -- 172.21.15.46:0/3917237573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 msgr2=0x56537cb5f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb5f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 -- 172.21.15.46:0/3917237573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537c7ce210 con 0x56537cb61cb0 2024-09-12T02:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 0x56537cb5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1e59c640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb5f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae1f59e640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fae18005850 tx=0x7fae1809bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cb6a92ec59256 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180704f0 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9555+0+0 (secure 0 0 0) 0x7fae18098730 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10012900 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae1809c0c0 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae28024740 10 client.15975 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56537c89a620 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.688+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10012900 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fae180a48a0 con 0x56537cb61cb0 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 20 client.15975 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cb7c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 4 client.15975 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537cac8a20 con 0x56537cb7a300 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cb7c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae02ff5640 10 client.15975 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.689+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cb7c710 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721a84f70db691af server_cookie=bf8e17cf954523ce in_seq=0 out_seq=0 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae02ff5640 1 -- 172.21.15.46:0/3917237573 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56537cac8a20 con 0x56537cb7a300 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae02ff5640 10 client.15975 handle_command_reply: tid=0 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 client.15975 shutdown 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 msgr2=0x56537cb5f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fae18005850 tx=0x7fae1809bb50 comp rx=0 tx=0).stop 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 shutdown_connections 2024-09-12T02:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cb7c710 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb61cb0 0x56537cb5f550 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb5f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae02ff5640 0 client.15975 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 --2- 172.21.15.46:0/3917237573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb56af0 0x56537cb5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 >> 172.21.15.46:0/3917237573 conn(0x56537cb53060 msgr2=0x56537cb58dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 shutdown_connections 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 1 -- 172.21.15.46:0/3917237573 wait complete. 2024-09-12T02:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.690+0000 7fae28024740 20 client.15975 trim_cache size 0 max 16384 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 Processor -- start 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 -- start start 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb82040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb82580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae201af690 con 0x56537cb56af0 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae201acf00 con 0x56537cb57410 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae28024740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae201aad70 con 0x56537cb61cb0 2024-09-12T02:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36006/0 (socket says 172.21.15.46:36006) 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.708+0000 7fae1ed9d640 1 -- 172.21.15.46:0/417339892 learned_addr learned my addr 172.21.15.46:0/417339892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1e59c640 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb82040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1f59e640 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb82580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 -- 172.21.15.46:0/417339892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 msgr2=0x56537cb82580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb82580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 -- 172.21.15.46:0/417339892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 msgr2=0x56537cb82040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb82040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 -- 172.21.15.46:0/417339892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537cb52e90 con 0x56537cb56af0 2024-09-12T02:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fae10009bb0 tx=0x7fae10005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5eb34c5d20ac7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae04ff9640 1 -- 172.21.15.46:0/417339892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10009760 con 0x56537cb56af0 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae04ff9640 1 -- 172.21.15.46:0/417339892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10012900 con 0x56537cb56af0 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.709+0000 7fae04ff9640 1 -- 172.21.15.46:0/417339892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10003850 con 0x56537cb56af0 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- 172.21.15.46:0/417339892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 msgr2=0x56537cb59560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fae10009bb0 tx=0x7fae10005b00 comp rx=0 tx=0).stop 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- 172.21.15.46:0/417339892 shutdown_connections 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb82580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb82040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 --2- 172.21.15.46:0/417339892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb59560 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- 172.21.15.46:0/417339892 >> 172.21.15.46:0/417339892 conn(0x56537cb53060 msgr2=0x56537cb60b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- 172.21.15.46:0/417339892 shutdown_connections 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- 172.21.15.46:0/417339892 wait complete. 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.710+0000 7fae28024740 1 -- start start 2024-09-12T02:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 Processor -- start 2024-09-12T02:58:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae201ab3c0 con 0x56537cb56af0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae201ac830 con 0x56537cb57410 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae201aba90 con 0x56537cb61cb0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae28024740 10 client.0 _ll_register_callbacks cb 0x7fae18098360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36022/0 (socket says 172.21.15.46:36022) 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 -- 172.21.15.46:0/4002891791 learned_addr learned my addr 172.21.15.46:0/4002891791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1e59c640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1f59e640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 -- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 msgr2=0x56537cb6efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb6efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 -- 172.21.15.46:0/4002891791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 msgr2=0x56537cb6ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb6ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1ed9d640 1 -- 172.21.15.46:0/4002891791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537cbe4210 con 0x56537cb56af0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1e59c640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.711+0000 7fae1f59e640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb6efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae1ed9d640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fae100119e0 tx=0x7fae100092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc699e0b2296cde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10005110 con 0x56537cb56af0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9555+0+0 (secure 0 0 0) 0x7fae100052b0 con 0x56537cb56af0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10012900 con 0x56537cb56af0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae10012450 con 0x56537cb56af0 2024-09-12T02:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 10 client.15981 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56537cbe54c0 con 0x56537cb56af0 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10012900 con 0x56537cb56af0 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fae1002d020 con 0x56537cb56af0 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 10 client.15981 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.712+0000 7fae28024740 20 client.15981 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cbe7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae28024740 4 client.15981 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56537cac2d70 con 0x56537cb7a300 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae1e59c640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cbe7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae02ff5640 10 client.15981 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.713+0000 7fae1e59c640 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cbe7950 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a6dc2feee9955b server_cookie=30be14645b6207d9 in_seq=0 out_seq=0 2024-09-12T02:58:06.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:06 smithi196 ceph-mon[34587]: pgmap v243: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-12T02:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.725+0000 7fae02ff5640 1 -- 172.21.15.46:0/4002891791 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56537cac2d70 con 0x56537cb7a300 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.725+0000 7fae02ff5640 10 client.15981 handle_command_reply: tid=0 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 client.15981 shutdown 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 msgr2=0x56537cb6e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fae100119e0 tx=0x7fae100092d0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 shutdown_connections 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56537cb7a300 0x56537cbe7950 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56537cb61cb0 0x56537cb6efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56537cb57410 0x56537cb6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae02ff5640 0 client.15981 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 --2- 172.21.15.46:0/4002891791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56537cb56af0 0x56537cb6e550 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 >> 172.21.15.46:0/4002891791 conn(0x56537cb53060 msgr2=0x56537cb519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 shutdown_connections 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 1 -- 172.21.15.46:0/4002891791 wait complete. 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.726+0000 7fae28024740 20 client.15981 trim_cache size 0 max 16384 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 msgr2=0x7fae080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 0x7fae080a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae14004650 tx=0x7fae14005780 comp rx=0 tx=0).stop 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 msgr2=0x7fae2019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae1800eb10 tx=0x7fae1800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 shutdown_connections 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fae080a2da0 0x7fae080a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae20107b70 0x7fae2019cb50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae20102470 0x7fae201987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 --2- 172.21.15.46:0/1233938736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae20101ad0 0x7fae20198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 >> 172.21.15.46:0/1233938736 conn(0x7fae200fd7d0 msgr2=0x7fae200fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 shutdown_connections 2024-09-12T02:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:06.727+0000 7fae251af640 1 -- 172.21.15.46:0/1233938736 wait complete. 2024-09-12T02:58:08.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:08 smithi046 ceph-mon[30951]: pgmap v244: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:08 smithi145 ceph-mon[34674]: pgmap v244: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:08 smithi196 ceph-mon[34587]: pgmap v244: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:10 smithi046 ceph-mon[30951]: pgmap v245: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:10.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:10 smithi046 ceph-mon[30951]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T02:58:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:10 smithi145 ceph-mon[34674]: pgmap v245: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:10 smithi145 ceph-mon[34674]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T02:58:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:10 smithi196 ceph-mon[34587]: pgmap v245: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 37 op/s 2024-09-12T02:58:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:10 smithi196 ceph-mon[34587]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T02:58:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:12 smithi046 ceph-mon[30951]: pgmap v247: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-12T02:58:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:12 smithi145 ceph-mon[34674]: pgmap v247: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-12T02:58:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:12 smithi196 ceph-mon[34587]: pgmap v247: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 10 op/s 2024-09-12T02:58:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:14 smithi046 ceph-mon[30951]: pgmap v248: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:14 smithi145 ceph-mon[34674]: pgmap v248: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:14 smithi196 ceph-mon[34587]: pgmap v248: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:16 smithi046 ceph-mon[30951]: pgmap v249: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:16 smithi145 ceph-mon[34674]: pgmap v249: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:16 smithi196 ceph-mon[34587]: pgmap v249: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:18 smithi046 ceph-mon[30951]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:18 smithi145 ceph-mon[34674]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:18 smithi196 ceph-mon[34587]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:19.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:19 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:19.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:19 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:19 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 -- 172.21.15.46:0/1754145485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201009e0 msgr2=0x7f6b20100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/1754145485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201009e0 0x7f6b20100e40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009930 tx=0x7f6b1402a420 comp rx=0 tx=0).stop 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 -- 172.21.15.46:0/1754145485 shutdown_connections 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/1754145485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 0x7f6b2010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/1754145485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201009e0 0x7f6b20100e40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/1754145485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201069f0 0x7f6b20106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.231+0000 7f6b25ca6640 1 -- 172.21.15.46:0/1754145485 >> 172.21.15.46:0/1754145485 conn(0x7f6b200fc6c0 msgr2=0x7f6b200feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.232+0000 7f6b25ca6640 1 -- 172.21.15.46:0/1754145485 shutdown_connections 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.232+0000 7f6b25ca6640 1 -- 172.21.15.46:0/1754145485 wait complete. 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.232+0000 7f6b25ca6640 1 Processor -- start 2024-09-12T02:58:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.232+0000 7f6b25ca6640 1 -- start start 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 0x7f6b20198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 0x7f6b20198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20110ce0 con 0x7f6b201069f0 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20110b60 con 0x7f6b201009e0 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b25ca6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20110e60 con 0x7f6b20101380 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44508/0 (socket says 172.21.15.46:44508) 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 -- 172.21.15.46:0/2575353586 learned_addr learned my addr 172.21.15.46:0/2575353586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1effd640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 0x7f6b20198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 0x7f6b20198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 -- 172.21.15.46:0/2575353586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 msgr2=0x7f6b20198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 0x7f6b20198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 -- 172.21.15.46:0/2575353586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 msgr2=0x7f6b20198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 0x7f6b20198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 -- 172.21.15.46:0/2575353586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2019d2f0 con 0x7f6b201069f0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 0x7f6b20198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.233+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800eaa0 tx=0x7f6b1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.234+0000 7f6b1cff9640 1 -- 172.21.15.46:0/2575353586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800cdb0 con 0x7f6b201069f0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.234+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2019d5e0 con 0x7f6b201069f0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.234+0000 7f6b1cff9640 1 -- 172.21.15.46:0/2575353586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b18005020 con 0x7f6b201069f0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.234+0000 7f6b1cff9640 1 -- 172.21.15.46:0/2575353586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180116d0 con 0x7f6b201069f0 2024-09-12T02:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.234+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a4e70 con 0x7f6b201069f0 2024-09-12T02:58:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.236+0000 7f6b1cff9640 1 -- 172.21.15.46:0/2575353586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b180051d0 con 0x7f6b201069f0 2024-09-12T02:58:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.237+0000 7f6b1cff9640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 0x7f6b080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.237+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 0x7f6b080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.237+0000 7f6b1cff9640 1 -- 172.21.15.46:0/2575353586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f6b180cfa20 con 0x7f6b201069f0 2024-09-12T02:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.237+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 0x7f6b080a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005fd0 tx=0x7f6b10005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 Processor -- start 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 -- start start 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910283ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b9101e82f0 con 0x55b910284410 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b910190270 con 0x55b91028ecb0 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9101e42b0 con 0x55b910283af0 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44522/0 (socket says 172.21.15.46:44522) 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b1effd640 1 -- 172.21.15.46:0/1812694398 learned_addr learned my addr 172.21.15.46:0/1812694398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.259+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910283ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 -- 172.21.15.46:0/1812694398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 msgr2=0x55b910283ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910283ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 -- 172.21.15.46:0/1812694398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 msgr2=0x55b9102910a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102910a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 -- 172.21.15.46:0/1812694398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91018f340 con 0x55b910284410 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b1effd640 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6b1402a940 tx=0x7f6b14005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c3dc5862605cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b05ffb640 1 -- 172.21.15.46:0/1812694398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14040070 con 0x55b910284410 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b05ffb640 1 -- 172.21.15.46:0/1812694398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b140316a0 con 0x55b910284410 2024-09-12T02:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.260+0000 7f6b05ffb640 1 -- 172.21.15.46:0/1812694398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035040 con 0x55b910284410 2024-09-12T02:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 -- 172.21.15.46:0/1812694398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 msgr2=0x55b91028e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6b1402a940 tx=0x7f6b14005c00 comp rx=0 tx=0).stop 2024-09-12T02:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 -- 172.21.15.46:0/1812694398 shutdown_connections 2024-09-12T02:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102910a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b91028e770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/1812694398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910283ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 -- 172.21.15.46:0/1812694398 >> 172.21.15.46:0/1812694398 conn(0x55b910280060 msgr2=0x55b910280440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 -- 172.21.15.46:0/1812694398 shutdown_connections 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.261+0000 7f6b28a3c740 1 -- 172.21.15.46:0/1812694398 wait complete. 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 -- start start 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 Processor -- start 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b910284410 0x55b91028c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b91028ecb0 0x55b91028c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b910291880 con 0x55b910283af0 2024-09-12T02:58:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b910171290 con 0x55b910284410 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b910291a00 con 0x55b91028ecb0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b28a3c740 10 client.0 _ll_register_callbacks cb 0x55b9101a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.262+0000 7f6b1f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44532/0 (socket says 172.21.15.46:44532) 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 -- 172.21.15.46:0/709234936 learned_addr learned my addr 172.21.15.46:0/709234936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b91028ecb0 0x55b91028c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1effd640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b910284410 0x55b91028c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 -- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b91028ecb0 msgr2=0x55b91028c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b91028ecb0 0x55b91028c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 -- 172.21.15.46:0/709234936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b910284410 msgr2=0x55b91028c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b910284410 0x55b91028c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 -- 172.21.15.46:0/709234936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90fefb210 con 0x55b910283af0 2024-09-12T02:58:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6b100116f0 tx=0x7f6b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c4d3995b2f17f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10004230 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f6b100043d0 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.263+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b140316a0 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b10030050 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 10 client.15993 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b90ffc7620 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b140316a0 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6b10003c10 con 0x55b910283af0 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 10 client.15993 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 20 client.15993 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b91029ec60 0x55b9102a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 4 client.15993 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9101f5a20 con 0x55b91029ec60 2024-09-12T02:58:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.264+0000 7f6b1effd640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b91029ec60 0x55b9102a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.265+0000 7f6b1effd640 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b91029ec60 0x55b9102a1050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9e853ac2022354 server_cookie=90367ce491ab6705 in_seq=0 out_seq=0 2024-09-12T02:58:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.265+0000 7f6afeffd640 10 client.15993 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.265+0000 7f6afeffd640 1 -- 172.21.15.46:0/709234936 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9101f5a20 con 0x55b91029ec60 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.265+0000 7f6afeffd640 10 client.15993 handle_command_reply: tid=0 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 client.15993 shutdown 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 msgr2=0x55b91028bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6b100116f0 tx=0x7f6b100046c0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 shutdown_connections 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b91029ec60 0x55b9102a1050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b91028ecb0 0x55b91028c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b910284410 0x55b91028c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/709234936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910283af0 0x55b91028bbf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 >> 172.21.15.46:0/709234936 conn(0x55b910280060 msgr2=0x55b91027f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6afeffd640 0 client.15993 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 shutdown_connections 2024-09-12T02:58:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 1 -- 172.21.15.46:0/709234936 wait complete. 2024-09-12T02:58:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.266+0000 7f6b28a3c740 20 client.15993 trim_cache size 0 max 16384 2024-09-12T02:58:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 Processor -- start 2024-09-12T02:58:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 -- start start 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b91028c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201b17f0 con 0x55b910284410 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201af060 con 0x55b91028ecb0 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201aced0 con 0x55b910283af0 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44536/0 (socket says 172.21.15.46:44536) 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b1f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b91028c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b1effd640 1 -- 172.21.15.46:0/3864495568 learned_addr learned my addr 172.21.15.46:0/3864495568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.288+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 -- 172.21.15.46:0/3864495568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 msgr2=0x55b91028c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b91028c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 -- 172.21.15.46:0/3864495568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 msgr2=0x55b9102a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 -- 172.21.15.46:0/3864495568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b910284240 con 0x55b910284410 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b91028c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55b910280ec0 tx=0x7f6b1402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b98b3941e1bbb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b057fa640 1 -- 172.21.15.46:0/3864495568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035040 con 0x55b910284410 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b057fa640 1 -- 172.21.15.46:0/3864495568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b14031840 con 0x55b910284410 2024-09-12T02:58:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.289+0000 7f6b057fa640 1 -- 172.21.15.46:0/3864495568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14040040 con 0x55b910284410 2024-09-12T02:58:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3864495568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 msgr2=0x55b9102a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55b910280ec0 tx=0x7f6b1402a420 comp rx=0 tx=0).stop 2024-09-12T02:58:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3864495568 shutdown_connections 2024-09-12T02:58:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b9102a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102a6940 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3864495568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b91028c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3864495568 >> 172.21.15.46:0/3864495568 conn(0x55b910280060 msgr2=0x55b91028dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3864495568 shutdown_connections 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.290+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3864495568 wait complete. 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 -- start start 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 Processor -- start 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910297540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b910293040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ad520 con 0x55b910284410 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ae990 con 0x55b91028ecb0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201adbf0 con 0x55b910283af0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b28a3c740 10 client.0 _ll_register_callbacks cb 0x7f6b1000be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b1effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44548/0 (socket says 172.21.15.46:44548) 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b1effd640 1 -- 172.21.15.46:0/3627953008 learned_addr learned my addr 172.21.15.46:0/3627953008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910297540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.291+0000 7f6b1ffff640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b910293040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 -- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 msgr2=0x55b910297540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910297540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 -- 172.21.15.46:0/3627953008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 msgr2=0x55b910293040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b910293040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 -- 172.21.15.46:0/3627953008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b910308910 con 0x55b910284410 2024-09-12T02:58:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b1effd640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b1402a940 tx=0x7f6b14005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b8d41f69ffea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1405d070 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f6b14005520 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b14031840 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b1406b050 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b28a3c740 10 client.15999 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.292+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b910309a80 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b14004c20 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6b14057070 con 0x55b910284410 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 20 client.15999 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b910309d00 0x55b91030c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 4 client.15999 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:58:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6b14031840 con 0x55b910309d00 2024-09-12T02:58:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.293+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b910309d00 0x55b91030c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.294+0000 7f6b1f7fe640 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b910309d00 0x55b91030c0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69aec1387965787b server_cookie=50eb8ef86e41d6ef in_seq=0 out_seq=0 2024-09-12T02:58:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.294+0000 7f6afeffd640 10 client.15999 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.302+0000 7f6afeffd640 1 -- 172.21.15.46:0/3627953008 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6b14031840 con 0x55b910309d00 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.302+0000 7f6afeffd640 10 client.15999 handle_command_reply: tid=0 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.302+0000 7f6b28a3c740 1 client.15999 shutdown 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.302+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 msgr2=0x55b9102928b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.302+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b1402a940 tx=0x7f6b14005e30 comp rx=0 tx=0).stop 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 shutdown_connections 2024-09-12T02:58:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b910309d00 0x55b91030c0f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b91028ecb0 0x55b910293040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b910284410 0x55b9102928b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 --2- 172.21.15.46:0/3627953008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b910283af0 0x55b910297540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 >> 172.21.15.46:0/3627953008 conn(0x55b910280060 msgr2=0x55b91028dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6afeffd640 0 client.15999 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 shutdown_connections 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 1 -- 172.21.15.46:0/3627953008 wait complete. 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b28a3c740 20 client.15999 trim_cache size 0 max 16384 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:58:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:58:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:58:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 msgr2=0x7f6b080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 0x7f6b080a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005fd0 tx=0x7f6b10005850 comp rx=0 tx=0).stop 2024-09-12T02:58:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 msgr2=0x7f6b2019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800eaa0 tx=0x7f6b1800ef60 comp rx=0 tx=0).stop 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.303+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 shutdown_connections 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6b080a2da0 0x7f6b080a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b201069f0 0x7f6b2019cbc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b20101380 0x7f6b20198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 --2- 172.21.15.46:0/2575353586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b201009e0 0x7f6b20198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 >> 172.21.15.46:0/2575353586 conn(0x7f6b200fc6c0 msgr2=0x7f6b2006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 shutdown_connections 2024-09-12T02:58:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:20.304+0000 7f6b25ca6640 1 -- 172.21.15.46:0/2575353586 wait complete. 2024-09-12T02:58:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:20 smithi046 ceph-mon[30951]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:20.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:20 smithi145 ceph-mon[34674]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:20 smithi196 ceph-mon[34587]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:58:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:21 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:21 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:21.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:21 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:58:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:21 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:21 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:21 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:58:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:21 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:21 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:21 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:58:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:22 smithi046 ceph-mon[30951]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:58:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:22 smithi145 ceph-mon[34674]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:58:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:22 smithi196 ceph-mon[34587]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:58:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:24 smithi046 ceph-mon[30951]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:58:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:24 smithi145 ceph-mon[34674]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:58:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:24 smithi196 ceph-mon[34587]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:58:26.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:26 smithi196 ceph-mon[34587]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:26 smithi046 ceph-mon[30951]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:26 smithi145 ceph-mon[34674]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:28 smithi046 ceph-mon[30951]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:28 smithi145 ceph-mon[34674]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:28 smithi196 ceph-mon[34587]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:30 smithi046 ceph-mon[30951]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:30 smithi145 ceph-mon[34674]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:30 smithi196 ceph-mon[34587]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:32.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:32 smithi046 ceph-mon[30951]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:32 smithi145 ceph-mon[34674]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:32 smithi196 ceph-mon[34587]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 -- 172.21.15.46:0/830364909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24101210 msgr2=0x7f3e2410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 --2- 172.21.15.46:0/830364909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24101210 0x7f3e2410ded0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009900 tx=0x7f3e1802a410 comp rx=0 tx=0).stop 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 -- 172.21.15.46:0/830364909 shutdown_connections 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 --2- 172.21.15.46:0/830364909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2410e410 0x7f3e24110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 --2- 172.21.15.46:0/830364909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24101210 0x7f3e2410ded0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 --2- 172.21.15.46:0/830364909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e24100860 0x7f3e24100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 -- 172.21.15.46:0/830364909 >> 172.21.15.46:0/830364909 conn(0x7f3e24077990 msgr2=0x7f3e240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 -- 172.21.15.46:0/830364909 shutdown_connections 2024-09-12T02:58:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.413+0000 7f3e28d91640 1 -- 172.21.15.46:0/830364909 wait complete. 2024-09-12T02:58:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 Processor -- start 2024-09-12T02:58:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 -- start start 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 0x7f3e2419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e2410e410 0x7f3e2419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24113f70 con 0x7f3e24100860 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24113df0 con 0x7f3e2410e410 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e28d91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241140f0 con 0x7f3e24101210 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.414+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47960/0 (socket says 172.21.15.46:47960) 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 -- 172.21.15.46:0/676794915 learned_addr learned my addr 172.21.15.46:0/676794915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e2410e410 0x7f3e2419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 0x7f3e2419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 -- 172.21.15.46:0/676794915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 msgr2=0x7f3e2419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 0x7f3e2419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 -- 172.21.15.46:0/676794915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e2410e410 msgr2=0x7f3e2419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e2410e410 0x7f3e2419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 -- 172.21.15.46:0/676794915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419f470 con 0x7f3e24100860 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 0x7f3e2419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e2259c640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9f0 tx=0x7f3e1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e137fe640 1 -- 172.21.15.46:0/676794915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400cd80 con 0x7f3e24100860 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e137fe640 1 -- 172.21.15.46:0/676794915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e140108e0 con 0x7f3e24100860 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419f760 con 0x7f3e24100860 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e137fe640 1 -- 172.21.15.46:0/676794915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14018920 con 0x7f3e24100860 2024-09-12T02:58:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.415+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a6ff0 con 0x7f3e24100860 2024-09-12T02:58:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.418+0000 7f3e137fe640 1 -- 172.21.15.46:0/676794915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3e14010a90 con 0x7f3e24100860 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 Processor -- start 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 -- start start 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e137fe640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 0x7f3e0c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 0x559d2b0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d2b0052f0 con 0x559d2b0a0af0 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559d2afad270 con 0x559d2b0abcb0 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d2b0012b0 con 0x559d2b0a1410 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 0x7f3e0c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47966/0 (socket says 172.21.15.46:47966) 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e2259c640 1 -- 172.21.15.46:0/1897981118 learned_addr learned my addr 172.21.15.46:0/1897981118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e137fe640 1 -- 172.21.15.46:0/676794915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f3e140cfae0 con 0x7f3e24100860 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.419+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 0x559d2b0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 -- 172.21.15.46:0/1897981118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 msgr2=0x559d2b0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 0x559d2b0ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 -- 172.21.15.46:0/1897981118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 msgr2=0x559d2b0ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 -- 172.21.15.46:0/1897981118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2afac340 con 0x559d2b0a0af0 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 0x559d2b0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e2259c640 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c6a0 tx=0x7f3e1401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ca2d038b7b2d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 0x7f3e0c0a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009fd0 tx=0x7f3e1c009d80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e10ff9640 1 -- 172.21.15.46:0/1897981118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1409c350 con 0x559d2b0a0af0 2024-09-12T02:58:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e10ff9640 1 -- 172.21.15.46:0/1897981118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1409bc50 con 0x559d2b0a0af0 2024-09-12T02:58:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.420+0000 7f3e10ff9640 1 -- 172.21.15.46:0/1897981118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e140cd100 con 0x559d2b0a0af0 2024-09-12T02:58:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/1897981118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 msgr2=0x559d2b0a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c6a0 tx=0x7f3e1401c850 comp rx=0 tx=0).stop 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/1897981118 shutdown_connections 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a1410 0x559d2b0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/1897981118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0a0ed0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/1897981118 >> 172.21.15.46:0/1897981118 conn(0x559d2b09d060 msgr2=0x559d2b09d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/1897981118 shutdown_connections 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.421+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/1897981118 wait complete. 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 -- start start 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 Processor -- start 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d2b0ae880 con 0x559d2b0abcb0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559d2af8e290 con 0x559d2b0a1410 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d2b0aea00 con 0x559d2b0a0af0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2bc0a740 10 client.0 _ll_register_callbacks cb 0x559d2afbf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47976/0 (socket says 172.21.15.46:47976) 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 -- 172.21.15.46:0/3072778644 learned_addr learned my addr 172.21.15.46:0/3072778644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e2259c640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 -- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 msgr2=0x559d2b0a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 -- 172.21.15.46:0/3072778644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 msgr2=0x559d2b0a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.422+0000 7f3e22d9d640 1 -- 172.21.15.46:0/3072778644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2ad18210 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00fdf0 tx=0x7f3e1c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c843a18e7fdf57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c013330 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f3e1c003e00 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1409bc50 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c0045b0 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e2bc0a740 10 client.16011 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d2ade4620 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1409bc50 con 0x559d2b0abcb0 2024-09-12T02:58:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.423+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e1c020050 con 0x559d2b0abcb0 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 10 client.16011 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 20 client.16011 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b0bbc70 0x559d2b0be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 4 client.16011 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d2b012a20 con 0x559d2b0bbc70 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2259c640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b0bbc70 0x559d2b0be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e09ffb640 10 client.16011 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.424+0000 7f3e2259c640 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b0bbc70 0x559d2b0be0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282287ac4568e82f server_cookie=3df2c99d255aa2df in_seq=0 out_seq=0 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e09ffb640 1 -- 172.21.15.46:0/3072778644 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d2b012a20 con 0x559d2b0bbc70 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e09ffb640 10 client.16011 handle_command_reply: tid=0 2024-09-12T02:58:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 client.16011 shutdown 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 msgr2=0x559d2b0a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00fdf0 tx=0x7f3e1c01d000 comp rx=0 tx=0).stop 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 shutdown_connections 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b0bbc70 0x559d2b0be0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0abcb0 0x559d2b0a96a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e09ffb640 0 client.16011 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/3072778644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 >> 172.21.15.46:0/3072778644 conn(0x559d2b09d060 msgr2=0x559d2b0a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 shutdown_connections 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/3072778644 wait complete. 2024-09-12T02:58:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.425+0000 7f3e2bc0a740 20 client.16011 trim_cache size 0 max 16384 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.445+0000 7f3e2bc0a740 1 Processor -- start 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.445+0000 7f3e2bc0a740 1 -- start start 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b3a60 con 0x559d2b0a1410 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b12d0 con 0x559d2b0abcb0 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241af140 con 0x559d2b0a0af0 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47984/0 (socket says 172.21.15.46:47984) 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 -- 172.21.15.46:0/95348383 learned_addr learned my addr 172.21.15.46:0/95348383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e2259c640 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 -- 172.21.15.46:0/95348383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 msgr2=0x559d2b0a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 -- 172.21.15.46:0/95348383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 msgr2=0x559d2b0a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 -- 172.21.15.46:0/95348383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2b09dcf0 con 0x559d2b0a1410 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.446+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3e18000c00 tx=0x7f3e18009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f515ec739059416b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e0bfff640 1 -- 172.21.15.46:0/95348383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18035070 con 0x559d2b0a1410 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e0bfff640 1 -- 172.21.15.46:0/95348383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1803f030 con 0x559d2b0a1410 2024-09-12T02:58:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e0bfff640 1 -- 172.21.15.46:0/95348383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18004db0 con 0x559d2b0a1410 2024-09-12T02:58:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/95348383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 msgr2=0x559d2b0a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3e18000c00 tx=0x7f3e18009680 comp rx=0 tx=0).stop 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/95348383 shutdown_connections 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0abcb0 0x559d2b0a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a1410 0x559d2b0a3560 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.447+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/95348383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0a0af0 0x559d2b0a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/95348383 >> 172.21.15.46:0/95348383 conn(0x559d2b09d060 msgr2=0x559d2b0aaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/95348383 shutdown_connections 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/95348383 wait complete. 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- start start 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 Processor -- start 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 0x559d2b0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241af790 con 0x559d2b0a0af0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b0c00 con 0x559d2b0a1410 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241afe60 con 0x559d2b0abcb0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.448+0000 7f3e2bc0a740 10 client.0 _ll_register_callbacks cb 0x7f3e1c013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e21d9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48000/0 (socket says 172.21.15.46:48000) 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 -- 172.21.15.46:0/2413467757 learned_addr learned my addr 172.21.15.46:0/2413467757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 0x559d2b0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 -- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 msgr2=0x559d2b0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 0x559d2b0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 -- 172.21.15.46:0/2413467757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 msgr2=0x559d2b0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 -- 172.21.15.46:0/2413467757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2b0aff80 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e22d9d640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 0x559d2b0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2259c640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3e1409b950 tx=0x7f3e14099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6cb8c9c5ff3256 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e140b1070 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7f3e14016b10 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1803f030 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e1401d210 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2bc0a740 10 client.16017 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.449+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d2b126750 con 0x559d2b0a0af0 2024-09-12T02:58:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e140b4e50 con 0x559d2b0a0af0 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e140b4020 con 0x559d2b0a0af0 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 10 client.16017 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 20 client.16017 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b126a40 0x559d2b128e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 4 client.16017 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d2b00cd70 con 0x559d2b126a40 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.450+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b126a40 0x559d2b128e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.451+0000 7f3e21d9b640 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b126a40 0x559d2b128e30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ffb431631fce4f server_cookie=615fe1523605c3d4 in_seq=0 out_seq=0 2024-09-12T02:58:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.451+0000 7f3e09ffb640 10 client.16017 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e09ffb640 1 -- 172.21.15.46:0/2413467757 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d2b00cd70 con 0x559d2b126a40 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e09ffb640 10 client.16017 handle_command_reply: tid=0 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e2bc0a740 1 client.16017 shutdown 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 msgr2=0x559d2b0b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3e1409b950 tx=0x7f3e14099ec0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.460+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 shutdown_connections 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559d2b126a40 0x559d2b128e30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d2b0abcb0 0x559d2b0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559d2b0a1410 0x559d2b0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e09ffb640 0 client.16017 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 --2- 172.21.15.46:0/2413467757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d2b0a0af0 0x559d2b0b4160 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 >> 172.21.15.46:0/2413467757 conn(0x559d2b09d060 msgr2=0x559d2b09b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 shutdown_connections 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 1 -- 172.21.15.46:0/2413467757 wait complete. 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e2bc0a740 20 client.16017 trim_cache size 0 max 16384 2024-09-12T02:58:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 msgr2=0x7f3e0c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 0x7f3e0c0a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009fd0 tx=0x7f3e1c009d80 comp rx=0 tx=0).stop 2024-09-12T02:58:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 msgr2=0x7f3e2419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9f0 tx=0x7f3e1400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 shutdown_connections 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3e0c0a2e20 0x7f3e0c0a52e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e2410e410 0x7f3e2419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e24101210 0x7f3e2419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 --2- 172.21.15.46:0/676794915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e24100860 0x7f3e2419a480 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.461+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 >> 172.21.15.46:0/676794915 conn(0x7f3e24077990 msgr2=0x7f3e240ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.462+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 shutdown_connections 2024-09-12T02:58:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:33.462+0000 7f3e28d91640 1 -- 172.21.15.46:0/676794915 wait complete. 2024-09-12T02:58:34.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:34 smithi046 ceph-mon[30951]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:34 smithi145 ceph-mon[34674]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:34 smithi196 ceph-mon[34587]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:36 smithi046 ceph-mon[30951]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:58:36.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:36 smithi145 ceph-mon[34674]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:58:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:36 smithi196 ceph-mon[34587]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 16 op/s 2024-09-12T02:58:38.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:38 smithi046 ceph-mon[30951]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:38 smithi145 ceph-mon[34674]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:38 smithi196 ceph-mon[34587]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-12T02:58:40.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:40 smithi046 ceph-mon[30951]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:40.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:40 smithi145 ceph-mon[34674]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:40 smithi196 ceph-mon[34587]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:42.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:42 smithi046 ceph-mon[30951]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:42.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:42 smithi145 ceph-mon[34674]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:42 smithi196 ceph-mon[34587]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:58:44.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:44 smithi046 ceph-mon[30951]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:44 smithi145 ceph-mon[34674]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:44.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:44 smithi196 ceph-mon[34587]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-09-12T02:58:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/2117295092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 msgr2=0x7fd8c8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/2117295092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8107f50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009930 tx=0x7fd8b802a420 comp rx=0 tx=0).stop 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/2117295092 shutdown_connections 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/2117295092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8102470 0x7fd8c810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/2117295092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8101ad0 0x7fd8c8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/2117295092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8107f50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/2117295092 >> 172.21.15.46:0/2117295092 conn(0x7fd8c80fd7d0 msgr2=0x7fd8c80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/2117295092 shutdown_connections 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/2117295092 wait complete. 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.557+0000 7fd8ce0d0640 1 Processor -- start 2024-09-12T02:58:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 -- start start 2024-09-12T02:58:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8101ad0 0x7fd8c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8102470 0x7fd8c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8111dd0 con 0x7fd8c8107b70 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8111c50 con 0x7fd8c8102470 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8ce0d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8111f50 con 0x7fd8c8101ad0 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8101ad0 0x7fd8c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54000/0 (socket says 172.21.15.46:54000) 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3567854834 learned_addr learned my addr 172.21.15.46:0/3567854834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8102470 0x7fd8c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3567854834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8101ad0 msgr2=0x7fd8c80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8101ad0 0x7fd8c80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3567854834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8102470 msgr2=0x7fd8c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8102470 0x7fd8c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.558+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3567854834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c81a6840 con 0x7fd8c8107b70 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000cad0 tx=0x7fd8c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8c4ff9640 1 -- 172.21.15.46:0/3567854834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000ae20 con 0x7fd8c8107b70 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8c4ff9640 1 -- 172.21.15.46:0/3567854834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c0005020 con 0x7fd8c8107b70 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81a6b30 con 0x7fd8c8107b70 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8c4ff9640 1 -- 172.21.15.46:0/3567854834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000f6b0 con 0x7fd8c8107b70 2024-09-12T02:58:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.559+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81a70d0 con 0x7fd8c8107b70 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.561+0000 7fd8c4ff9640 1 -- 172.21.15.46:0/3567854834 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd8c00051d0 con 0x7fd8c8107b70 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8c4ff9640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 0x7fd8b00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 Processor -- start 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8c4ff9640 1 -- 172.21.15.46:0/3567854834 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7fd8c00cf650 con 0x7fd8c8107b70 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 -- start start 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 0x7fd8b00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d81eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d82c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56258d7832f0 con 0x56258d81f410 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56258d72b270 con 0x56258d829cb0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56258d77f2b0 con 0x56258d81eaf0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.562+0000 7fd8c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54014/0 (socket says 172.21.15.46:54014) 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 -- 172.21.15.46:0/1313802144 learned_addr learned my addr 172.21.15.46:0/1313802144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 0x7fd8b00a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b8000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d81eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d82c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 -- 172.21.15.46:0/1313802144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 msgr2=0x56258d81eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d81eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 -- 172.21.15.46:0/1313802144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 msgr2=0x56258d82c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d82c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 -- 172.21.15.46:0/1313802144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258d72a340 con 0x56258d81f410 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d82c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bcb0 tx=0x7fd8bc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d12f1c8594f2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8adffb640 1 -- 172.21.15.46:0/1313802144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004030 con 0x56258d81f410 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8adffb640 1 -- 172.21.15.46:0/1313802144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8bc0041d0 con 0x56258d81f410 2024-09-12T02:58:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.563+0000 7fd8adffb640 1 -- 172.21.15.46:0/1313802144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0116a0 con 0x56258d81f410 2024-09-12T02:58:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1313802144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 msgr2=0x56258d829770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00bcb0 tx=0x7fd8bc0107a0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1313802144 shutdown_connections 2024-09-12T02:58:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d82c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81f410 0x56258d829770 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1313802144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d81eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1313802144 >> 172.21.15.46:0/1313802144 conn(0x56258d81b060 msgr2=0x56258d81b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1313802144 shutdown_connections 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.564+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1313802144 wait complete. 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 -- start start 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 Processor -- start 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d826c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d827150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56258d82c880 con 0x56258d829cb0 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56258d70c290 con 0x56258d81f410 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56258d82ca00 con 0x56258d81eaf0 2024-09-12T02:58:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8d0e65740 10 client.0 _ll_register_callbacks cb 0x56258d73d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.565+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54020/0 (socket says 172.21.15.46:54020) 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3210007572 learned_addr learned my addr 172.21.15.46:0/3210007572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d827150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d826c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 msgr2=0x56258d826c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d826c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3210007572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 msgr2=0x56258d827150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d827150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 -- 172.21.15.46:0/3210007572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258d496210 con 0x56258d829cb0 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d827150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0098240 tx=0x7fd8c009b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd1010365f1e775 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c001d040 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7fd8c00860c0 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8bc0041d0 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd8c00a4430 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8d0e65740 10 client.16029 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.566+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56258d562620 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8bc0041d0 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8c00afce0 con 0x56258d829cb0 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 10 client.16029 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 20 client.16029 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d839c80 0x56258d83c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 4 client.16029 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258d790a20 con 0x56258d839c80 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.567+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d839c80 0x56258d83c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.568+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d839c80 0x56258d83c070 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a201a2e5aada8fb server_cookie=42a5ea302acd8cd6 in_seq=0 out_seq=0 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.568+0000 7fd8a6ffd640 10 client.16029 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.568+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/3210007572 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56258d790a20 con 0x56258d839c80 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.568+0000 7fd8a6ffd640 10 client.16029 handle_command_reply: tid=0 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.568+0000 7fd8d0e65740 1 client.16029 shutdown 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 msgr2=0x56258d827690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0098240 tx=0x7fd8c009b2c0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 shutdown_connections 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d839c80 0x56258d83c070 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d829cb0 0x56258d827690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d827150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3210007572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81eaf0 0x56258d826c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 >> 172.21.15.46:0/3210007572 conn(0x56258d81b060 msgr2=0x56258d81a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8a6ffd640 0 client.16029 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 shutdown_connections 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3210007572 wait complete. 2024-09-12T02:58:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.569+0000 7fd8d0e65740 20 client.16029 trim_cache size 0 max 16384 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 Processor -- start 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 -- start start 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d841940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d829cb0 0x56258d841e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81b3b40 con 0x56258d81eaf0 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81b13b0 con 0x56258d81f410 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81af220 con 0x56258d829cb0 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54022/0 (socket says 172.21.15.46:54022) 2024-09-12T02:58:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 -- 172.21.15.46:0/3612731255 learned_addr learned my addr 172.21.15.46:0/3612731255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d841940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c7fff640 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d829cb0 0x56258d841e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 -- 172.21.15.46:0/3612731255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d829cb0 msgr2=0x56258d841e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d829cb0 0x56258d841e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 -- 172.21.15.46:0/3612731255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 msgr2=0x56258d841940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d841940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.586+0000 7fd8c77fe640 1 -- 172.21.15.46:0/3612731255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258d81c150 con 0x56258d81eaf0 2024-09-12T02:58:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.587+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009930 tx=0x7fd8b803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b20b3de33a09e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.587+0000 7fd8ad7fa640 1 -- 172.21.15.46:0/3612731255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8057070 con 0x56258d81eaf0 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.587+0000 7fd8ad7fa640 1 -- 172.21.15.46:0/3612731255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b8003aa0 con 0x56258d81eaf0 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.587+0000 7fd8ad7fa640 1 -- 172.21.15.46:0/3612731255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8035040 con 0x56258d81eaf0 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3612731255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 msgr2=0x56258d8274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009930 tx=0x7fd8b803d000 comp rx=0 tx=0).stop 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3612731255 shutdown_connections 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d829cb0 0x56258d841e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d81f410 0x56258d841940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/3612731255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d8274c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3612731255 >> 172.21.15.46:0/3612731255 conn(0x56258d81b060 msgr2=0x56258d8254c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3612731255 shutdown_connections 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- 172.21.15.46:0/3612731255 wait complete. 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 -- start start 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.588+0000 7fd8d0e65740 1 Processor -- start 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81f410 0x56258d823920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d823e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81af870 con 0x56258d81eaf0 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81b0ce0 con 0x56258d829cb0 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81aff40 con 0x56258d81f410 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8d0e65740 10 client.0 _ll_register_callbacks cb 0x7fd8c000ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54030/0 (socket says 172.21.15.46:54030) 2024-09-12T02:58:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 -- 172.21.15.46:0/1094311288 learned_addr learned my addr 172.21.15.46:0/1094311288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81f410 0x56258d823920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 -- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81f410 msgr2=0x56258d823920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81f410 0x56258d823920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 -- 172.21.15.46:0/1094311288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 msgr2=0x56258d823e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d823e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 -- 172.21.15.46:0/1094311288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258d824640 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.589+0000 7fd8c77fe640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8031e40 tx=0x7fd8b8031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d216f6c67cc879 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8057070 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b8003aa0 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7fd8b8003c50 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd8b804b440 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 10 client.16035 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56258d8a4900 con 0x56258d81eaf0 2024-09-12T02:58:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b8003aa0 con 0x56258d81eaf0 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8b8063030 con 0x56258d81eaf0 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 10 client.16035 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 20 client.16035 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d8a5020 0x56258d8a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 4 client.16035 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.590+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56258d78ad70 con 0x56258d8a5020 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.591+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d8a5020 0x56258d8a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.591+0000 7fd8c6ffd640 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d8a5020 0x56258d8a7440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7f9ba4158aa57c server_cookie=14172f3c2bdfb19a in_seq=0 out_seq=0 2024-09-12T02:58:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.591+0000 7fd8a6ffd640 10 client.16035 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8a6ffd640 1 -- 172.21.15.46:0/1094311288 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56258d78ad70 con 0x56258d8a5020 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8a6ffd640 10 client.16035 handle_command_reply: tid=0 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 client.16035 shutdown 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 msgr2=0x56258d832440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8031e40 tx=0x7fd8b8031a00 comp rx=0 tx=0).stop 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 shutdown_connections 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56258d8a5020 0x56258d8a7440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56258d829cb0 0x56258d823e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56258d81f410 0x56258d823920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8a6ffd640 0 client.16035 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 --2- 172.21.15.46:0/1094311288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56258d81eaf0 0x56258d832440 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 >> 172.21.15.46:0/1094311288 conn(0x56258d81b060 msgr2=0x56258d8199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 shutdown_connections 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 1 -- 172.21.15.46:0/1094311288 wait complete. 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.604+0000 7fd8d0e65740 20 client.16035 trim_cache size 0 max 16384 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:58:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 msgr2=0x7fd8b00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 0x7fd8b00a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 msgr2=0x7fd8c8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000cad0 tx=0x7fd8c000cf90 comp rx=0 tx=0).stop 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 shutdown_connections 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd8b00a2e20 0x7fd8b00a52e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c8107b70 0x7fd8c8074f70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8102470 0x7fd8c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 --2- 172.21.15.46:0/3567854834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8c8101ad0 0x7fd8c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 >> 172.21.15.46:0/3567854834 conn(0x7fd8c80fd7d0 msgr2=0x7fd8c80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 shutdown_connections 2024-09-12T02:58:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:46.605+0000 7fd8ce0d0640 1 -- 172.21.15.46:0/3567854834 wait complete. 2024-09-12T02:58:46.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:46 smithi145 ceph-mon[34674]: pgmap v264: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:46 smithi196 ceph-mon[34587]: pgmap v264: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:46.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:46 smithi046 ceph-mon[30951]: pgmap v264: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:58:48.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:48 smithi145 ceph-mon[34674]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:58:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:48 smithi196 ceph-mon[34587]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:58:48.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:48 smithi046 ceph-mon[30951]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:58:50.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:50 smithi145 ceph-mon[34674]: pgmap v266: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:50 smithi196 ceph-mon[34587]: pgmap v266: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:50.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:50 smithi046 ceph-mon[30951]: pgmap v266: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:52 smithi145 ceph-mon[34674]: pgmap v267: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:52 smithi196 ceph-mon[34587]: pgmap v267: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:52.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:52 smithi046 ceph-mon[30951]: pgmap v267: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-09-12T02:58:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:54 smithi145 ceph-mon[34674]: pgmap v268: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:54 smithi196 ceph-mon[34587]: pgmap v268: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:54 smithi046 ceph-mon[30951]: pgmap v268: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:56 smithi145 ceph-mon[34674]: pgmap v269: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-12T02:58:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:56 smithi196 ceph-mon[34587]: pgmap v269: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-12T02:58:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:56 smithi046 ceph-mon[30951]: pgmap v269: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-12T02:58:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:58:58 smithi145 ceph-mon[34674]: pgmap v270: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:58:58 smithi196 ceph-mon[34587]: pgmap v270: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:58:58 smithi046 ceph-mon[30951]: pgmap v270: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-12T02:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 -- 172.21.15.46:0/3266638075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa44101350 msgr2=0x7ffa4410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 --2- 172.21.15.46:0/3266638075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa44101350 0x7ffa4410d880 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009930 tx=0x7ffa3402a420 comp rx=0 tx=0).stop 2024-09-12T02:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 -- 172.21.15.46:0/3266638075 shutdown_connections 2024-09-12T02:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 --2- 172.21.15.46:0/3266638075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa44101350 0x7ffa4410d880 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 --2- 172.21.15.46:0/3266638075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441009b0 0x7ffa44100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 --2- 172.21.15.46:0/3266638075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa441069c0 0x7ffa44106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.700+0000 7ffa48e82640 1 -- 172.21.15.46:0/3266638075 >> 172.21.15.46:0/3266638075 conn(0x7ffa44077990 msgr2=0x7ffa440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.701+0000 7ffa48e82640 1 -- 172.21.15.46:0/3266638075 shutdown_connections 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.701+0000 7ffa48e82640 1 -- 172.21.15.46:0/3266638075 wait complete. 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.701+0000 7ffa48e82640 1 Processor -- start 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.701+0000 7ffa48e82640 1 -- start start 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa44101350 0x7ffa441973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441069c0 0x7ffa44197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110d00 con 0x7ffa441009b0 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110b80 con 0x7ffa44101350 2024-09-12T02:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa48e82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110e80 con 0x7ffa441069c0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56708/0 (socket says 172.21.15.46:56708) 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 -- 172.21.15.46:0/4054711443 learned_addr learned my addr 172.21.15.46:0/4054711443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441069c0 0x7ffa44197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa44101350 0x7ffa441973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 -- 172.21.15.46:0/4054711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441069c0 msgr2=0x7ffa44197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441069c0 0x7ffa44197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 -- 172.21.15.46:0/4054711443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa44101350 msgr2=0x7ffa441973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa44101350 0x7ffa441973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 -- 172.21.15.46:0/4054711443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441a6700 con 0x7ffa441009b0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.702+0000 7ffa4259c640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00ebc0 tx=0x7ffa3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.703+0000 7ffa3b7fe640 1 -- 172.21.15.46:0/4054711443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c010630 con 0x7ffa441009b0 2024-09-12T02:58:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.703+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441a69f0 con 0x7ffa441009b0 2024-09-12T02:58:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.703+0000 7ffa3b7fe640 1 -- 172.21.15.46:0/4054711443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c010c80 con 0x7ffa441009b0 2024-09-12T02:58:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.703+0000 7ffa3b7fe640 1 -- 172.21.15.46:0/4054711443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c018b30 con 0x7ffa441009b0 2024-09-12T02:58:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.703+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441a6f10 con 0x7ffa441009b0 2024-09-12T02:58:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.705+0000 7ffa3b7fe640 1 -- 172.21.15.46:0/4054711443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa3c018cd0 con 0x7ffa441009b0 2024-09-12T02:58:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 Processor -- start 2024-09-12T02:58:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 -- start start 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa3b7fe640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 0x7ffa300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 0x7ffa300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa3b7fe640 1 -- 172.21.15.46:0/4054711443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7ffa3c0cfd10 con 0x7ffa441009b0 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8341c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8341f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c833762f0 con 0x562c83411af0 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562c8331e270 con 0x562c83412410 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c833722b0 con 0x562c8341ccb0 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56716/0 (socket says 172.21.15.46:56716) 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa4259c640 1 -- 172.21.15.46:0/2528459579 learned_addr learned my addr 172.21.15.46:0/2528459579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.706+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8341c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 -- 172.21.15.46:0/2528459579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 msgr2=0x562c8341f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8341f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 -- 172.21.15.46:0/2528459579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 msgr2=0x562c83411ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 -- 172.21.15.46:0/2528459579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8331d340 con 0x562c83412410 2024-09-12T02:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa4259c640 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 0x7ffa300a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005ab0 tx=0x7ffa34031400 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8341c770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009bd0 tx=0x7ffa2c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd02ad6b019dc56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa38ff9640 1 -- 172.21.15.46:0/2528459579 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c004230 con 0x562c83412410 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa38ff9640 1 -- 172.21.15.46:0/2528459579 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa2c009e40 con 0x562c83412410 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.707+0000 7ffa38ff9640 1 -- 172.21.15.46:0/2528459579 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c004c60 con 0x562c83412410 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- 172.21.15.46:0/2528459579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 msgr2=0x562c8341c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8341c770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009bd0 tx=0x7ffa2c011810 comp rx=0 tx=0).stop 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- 172.21.15.46:0/2528459579 shutdown_connections 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8341f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8341c770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/2528459579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c83411ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- 172.21.15.46:0/2528459579 >> 172.21.15.46:0/2528459579 conn(0x562c8340e060 msgr2=0x562c8340e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- 172.21.15.46:0/2528459579 shutdown_connections 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- 172.21.15.46:0/2528459579 wait complete. 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 -- start start 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.708+0000 7ffa4bc22740 1 Processor -- start 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c83419b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c8341a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c8341f880 con 0x562c8341ccb0 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562c832ff290 con 0x562c83411af0 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c8341fa00 con 0x562c83412410 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4bc22740 10 client.0 _ll_register_callbacks cb 0x562c83330140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56722/0 (socket says 172.21.15.46:56722) 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3249858389 learned_addr learned my addr 172.21.15.46:0/3249858389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c8341a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c83419b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 msgr2=0x562c8341a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c8341a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3249858389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 msgr2=0x562c83419b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c83419b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3249858389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c83089210 con 0x562c8341ccb0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009930 tx=0x7ffa3402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb149b303f85303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.709+0000 7ffa29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3404d070 con 0x562c8341ccb0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7ffa340316a0 con 0x562c8341ccb0 2024-09-12T02:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c83383a20 con 0x562c8341ccb0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa34059050 con 0x562c8341ccb0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 10 client.16047 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c83155620 con 0x562c8341ccb0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562c83383a20 con 0x562c8341ccb0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ffa34049940 con 0x562c8341ccb0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 10 client.16047 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 20 client.16047 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c83435380 0x562c83437770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 4 client.16047 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.710+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8337dd70 con 0x562c83435380 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.711+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c83435380 0x562c83437770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.711+0000 7ffa29ffb640 10 client.16047 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.711+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c83435380 0x562c83437770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b72d275d0e3a654 server_cookie=ded167f590be6428 in_seq=0 out_seq=0 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3249858389 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c8337dd70 con 0x562c83435380 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa29ffb640 10 client.16047 handle_command_reply: tid=0 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 client.16047 shutdown 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 msgr2=0x562c8341a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009930 tx=0x7ffa3402a420 comp rx=0 tx=0).stop 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 shutdown_connections 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c83435380 0x562c83437770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c8341a5e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c8341a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa29ffb640 0 client.16047 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3249858389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c83419b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 >> 172.21.15.46:0/3249858389 conn(0x562c8340e060 msgr2=0x562c83413dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 shutdown_connections 2024-09-12T02:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3249858389 wait complete. 2024-09-12T02:58:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.712+0000 7ffa4bc22740 20 client.16047 trim_cache size 0 max 16384 2024-09-12T02:58:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 Processor -- start 2024-09-12T02:58:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 -- start start 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8343d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8343d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441af5b0 con 0x562c83411af0 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ace20 con 0x562c83412410 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441aacd0 con 0x562c8341ccb0 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56730/0 (socket says 172.21.15.46:56730) 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa4259c640 1 -- 172.21.15.46:0/1711368566 learned_addr learned my addr 172.21.15.46:0/1711368566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8343d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.729+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8343d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 -- 172.21.15.46:0/1711368566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 msgr2=0x562c8343d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8343d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 -- 172.21.15.46:0/1711368566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 msgr2=0x562c8343d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8343d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 -- 172.21.15.46:0/1711368566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c83419930 con 0x562c83411af0 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa4259c640 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0cd050 tx=0x7ffa3c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed993dae1919571 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa2bfff640 1 -- 172.21.15.46:0/1711368566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0b8070 con 0x562c83411af0 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa2bfff640 1 -- 172.21.15.46:0/1711368566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c099fd0 con 0x562c83411af0 2024-09-12T02:58:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.730+0000 7ffa2bfff640 1 -- 172.21.15.46:0/1711368566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c09b380 con 0x562c83411af0 2024-09-12T02:58:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- 172.21.15.46:0/1711368566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 msgr2=0x562c8341bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0cd050 tx=0x7ffa3c01a040 comp rx=0 tx=0).stop 2024-09-12T02:58:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- 172.21.15.46:0/1711368566 shutdown_connections 2024-09-12T02:58:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c8341ccb0 0x562c8343d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83412410 0x562c8343d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/1711368566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c83411af0 0x562c8341bad0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- 172.21.15.46:0/1711368566 >> 172.21.15.46:0/1711368566 conn(0x562c8340e060 msgr2=0x562c8341b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- 172.21.15.46:0/1711368566 shutdown_connections 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- 172.21.15.46:0/1711368566 wait complete. 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 -- start start 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.731+0000 7ffa4bc22740 1 Processor -- start 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c8342db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c83428e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ab320 con 0x562c8341ccb0 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ac750 con 0x562c83411af0 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ab9b0 con 0x562c83412410 2024-09-12T02:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4bc22740 10 client.0 _ll_register_callbacks cb 0x7ffa2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56742/0 (socket says 172.21.15.46:56742) 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3528539557 learned_addr learned my addr 172.21.15.46:0/3528539557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa41d9b640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c83428e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c8342db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 msgr2=0x562c83428e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c83428e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3528539557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 msgr2=0x562c8342db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c8342db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 -- 172.21.15.46:0/3528539557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c83429a10 con 0x562c8341ccb0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa42d9d640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005a80 tx=0x7ffa34049670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ea7d918a0df07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.732+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34062060 con 0x562c8341ccb0 2024-09-12T02:58:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9467+0+0 (secure 0 0 0) 0x7ffa340314a0 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3c099fd0 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa34069050 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 10 client.16053 fetch_fsmap learned FSMap version 31 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c8349ff40 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c099fd0 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ffa3403c440 con 0x562c8341ccb0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 10 client.16053 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 20 client.16053 populate_metadata read hostname 'smithi046' 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c834a0690 0x562c834a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 4 client.16053 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.733+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c832fa900 con 0x562c834a0690 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.734+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c834a0690 0x562c834a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.734+0000 7ffa4259c640 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c834a0690 0x562c834a2ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbe9bc976d4fd36 server_cookie=ee2791ff17297722 in_seq=0 out_seq=0 2024-09-12T02:58:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.734+0000 7ffa29ffb640 10 client.16053 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.745+0000 7ffa29ffb640 1 -- 172.21.15.46:0/3528539557 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c832fa900 con 0x562c834a0690 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.745+0000 7ffa29ffb640 10 client.16053 handle_command_reply: tid=0 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.745+0000 7ffa4bc22740 1 client.16053 shutdown 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.745+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 msgr2=0x562c83429370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.745+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005a80 tx=0x7ffa34049670 comp rx=0 tx=0).stop 2024-09-12T02:58:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 shutdown_connections 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562c834a0690 0x562c834a2ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c8341ccb0 0x562c83429370 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c83412410 0x562c83428e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa29ffb640 0 client.16053 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 --2- 172.21.15.46:0/3528539557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562c83411af0 0x562c8342db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 >> 172.21.15.46:0/3528539557 conn(0x562c8340e060 msgr2=0x562c8341a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 shutdown_connections 2024-09-12T02:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 1 -- 172.21.15.46:0/3528539557 wait complete. 2024-09-12T02:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa4bc22740 20 client.16053 trim_cache size 0 max 16384 2024-09-12T02:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:58:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:58:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:58:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:58:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 msgr2=0x7ffa300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 0x7ffa300a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005ab0 tx=0x7ffa34031400 comp rx=0 tx=0).stop 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 msgr2=0x7ffa44196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00ebc0 tx=0x7ffa3c00c6a0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 shutdown_connections 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffa300a2da0 0x7ffa300a5260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa441069c0 0x7ffa44197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa44101350 0x7ffa441973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 --2- 172.21.15.46:0/4054711443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa441009b0 0x7ffa44196e90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.746+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 >> 172.21.15.46:0/4054711443 conn(0x7ffa44077990 msgr2=0x7ffa441037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.747+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 shutdown_connections 2024-09-12T02:58:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:58:59.747+0000 7ffa48e82640 1 -- 172.21.15.46:0/4054711443 wait complete. 2024-09-12T02:59:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:00 smithi145 ceph-mon[34674]: pgmap v271: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:00 smithi196 ceph-mon[34587]: pgmap v271: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:00.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:00 smithi046 ceph-mon[30951]: pgmap v271: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:01.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:01 smithi145 ceph-mon[34674]: pgmap v272: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:01 smithi196 ceph-mon[34587]: pgmap v272: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:01.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:01 smithi046 ceph-mon[30951]: pgmap v272: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:03.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:03 smithi145 ceph-mon[34674]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T02:59:03.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:03 smithi145 ceph-mon[34674]: pgmap v274: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:03 smithi196 ceph-mon[34587]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T02:59:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:03 smithi196 ceph-mon[34587]: pgmap v274: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:03.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:03 smithi046 ceph-mon[30951]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T02:59:03.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:03 smithi046 ceph-mon[30951]: pgmap v274: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:05.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:05 smithi196 ceph-mon[34587]: pgmap v275: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:06.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:05 smithi046 ceph-mon[30951]: pgmap v275: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:05 smithi145 ceph-mon[34674]: pgmap v275: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:07.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:07 smithi196 ceph-mon[34587]: pgmap v276: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:08.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:07 smithi046 ceph-mon[30951]: pgmap v276: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:07 smithi145 ceph-mon[34674]: pgmap v276: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:09.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:09 smithi196 ceph-mon[34587]: pgmap v277: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:10.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:09 smithi046 ceph-mon[30951]: pgmap v277: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:10.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:09 smithi145 ceph-mon[34674]: pgmap v277: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-12T02:59:11.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:11 smithi046 ceph-mon[30951]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T02:59:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:11 smithi145 ceph-mon[34674]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T02:59:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:11 smithi196 ceph-mon[34587]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T02:59:12.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:12 smithi046 ceph-mon[30951]: pgmap v279: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-12T02:59:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:12 smithi145 ceph-mon[34674]: pgmap v279: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-12T02:59:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:12 smithi196 ceph-mon[34587]: pgmap v279: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 11 op/s 2024-09-12T02:59:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 -- 172.21.15.46:0/2492469065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0010c0d0 msgr2=0x7fdc0010e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 --2- 172.21.15.46:0/2492469065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0010c0d0 0x7fdc0010e570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009ca0 tx=0x7fdbf802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 -- 172.21.15.46:0/2492469065 shutdown_connections 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 --2- 172.21.15.46:0/2492469065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0010c0d0 0x7fdc0010e570 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 --2- 172.21.15.46:0/2492469065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc001035c0 0x7fdc0010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 --2- 172.21.15.46:0/2492469065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc00102ca0 0x7fdc00103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 -- 172.21.15.46:0/2492469065 >> 172.21.15.46:0/2492469065 conn(0x7fdc000fc6b0 msgr2=0x7fdc000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 -- 172.21.15.46:0/2492469065 shutdown_connections 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.853+0000 7fdc076ba640 1 -- 172.21.15.46:0/2492469065 wait complete. 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 Processor -- start 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 -- start start 2024-09-12T02:59:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc001035c0 0x7fdc0019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc0010c0d0 0x7fdc0019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00111d80 con 0x7fdc00102ca0 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00111c00 con 0x7fdc0010c0d0 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc076ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00111f00 con 0x7fdc001035c0 2024-09-12T02:59:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43182/0 (socket says 172.21.15.46:43182) 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc05456640 1 -- 172.21.15.46:0/1307430290 learned_addr learned my addr 172.21.15.46:0/1307430290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc04c55640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc001035c0 0x7fdc0019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.854+0000 7fdc05c57640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc0010c0d0 0x7fdc0019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 -- 172.21.15.46:0/1307430290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc001035c0 msgr2=0x7fdc0019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc001035c0 0x7fdc0019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 -- 172.21.15.46:0/1307430290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc0010c0d0 msgr2=0x7fdc0019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc0010c0d0 0x7fdc0019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 -- 172.21.15.46:0/1307430290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0019f3d0 con 0x7fdc00102ca0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc05456640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c890 tx=0x7fdbfc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdbf67fc640 1 -- 172.21.15.46:0/1307430290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc004220 con 0x7fdc00102ca0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdbf67fc640 1 -- 172.21.15.46:0/1307430290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbfc0043c0 con 0x7fdc00102ca0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0019f6c0 con 0x7fdc00102ca0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdbf67fc640 1 -- 172.21.15.46:0/1307430290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc004a00 con 0x7fdc00102ca0 2024-09-12T02:59:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.855+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a6f50 con 0x7fdc00102ca0 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.858+0000 7fdbf67fc640 1 -- 172.21.15.46:0/1307430290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdbfc004ba0 con 0x7fdc00102ca0 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.858+0000 7fdc0a45a740 1 Processor -- start 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.858+0000 7fdc0a45a740 1 -- start start 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e68ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e73770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562f45dcd2f0 con 0x562f45e69410 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562f45d75270 con 0x562f45e73cb0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f45dc92b0 con 0x562f45e68af0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdbf67fc640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 0x7fdbec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05c57640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 0x7fdbec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e73770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdbf67fc640 1 -- 172.21.15.46:0/1307430290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fdbfc0cf410 con 0x7fdc00102ca0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e73770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43194/0 (socket says 172.21.15.46:43194) 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05456640 1 -- 172.21.15.46:0/3567766994 learned_addr learned my addr 172.21.15.46:0/3567766994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05c57640 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e68ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 -- 172.21.15.46:0/3567766994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 msgr2=0x562f45e68ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e68ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 -- 172.21.15.46:0/3567766994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 msgr2=0x562f45e73770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e73770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc04c55640 1 -- 172.21.15.46:0/3567766994 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f45d74340 con 0x562f45e73cb0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.859+0000 7fdc05c57640 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e68ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e760a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc000697f0 tx=0x7fdbf00103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57163f500ed59d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdc05c57640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 0x7fdbec0a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8005e30 tx=0x7fdbf80093d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdbeb7fe640 1 -- 172.21.15.46:0/3567766994 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0010e90 con 0x562f45e73cb0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdbeb7fe640 1 -- 172.21.15.46:0/3567766994 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbf0005020 con 0x562f45e73cb0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdbeb7fe640 1 -- 172.21.15.46:0/3567766994 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0004b10 con 0x562f45e73cb0 2024-09-12T02:59:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3567766994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 msgr2=0x562f45e760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.860+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e760a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc000697f0 tx=0x7fdbf00103d0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3567766994 shutdown_connections 2024-09-12T02:59:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e760a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3567766994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e68ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3567766994 >> 172.21.15.46:0/3567766994 conn(0x562f45e65060 msgr2=0x562f45e65440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3567766994 shutdown_connections 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3567766994 wait complete. 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- start start 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 Processor -- start 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e71040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e73cb0 0x562f45e71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562f45e76880 con 0x562f45e68af0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562f45d56290 con 0x562f45e69410 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.861+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f45e76a00 con 0x562f45e73cb0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc0a45a740 10 client.0 _ll_register_callbacks cb 0x562f45d87140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43208/0 (socket says 172.21.15.46:43208) 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 -- 172.21.15.46:0/4141400792 learned_addr learned my addr 172.21.15.46:0/4141400792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05c57640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e73cb0 0x562f45e71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc04c55640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e71040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 -- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e73cb0 msgr2=0x562f45e71580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e73cb0 0x562f45e71580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 -- 172.21.15.46:0/4141400792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 msgr2=0x562f45e71040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e71040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 -- 172.21.15.46:0/4141400792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f45ae0210 con 0x562f45e68af0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc04c55640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc05456640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc09b5c0 tx=0x7fdbfc066700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaabd14941c69cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc01d040 con 0x562f45e68af0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fdbfc0cf150 con 0x562f45e68af0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.862+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf0005020 con 0x562f45e68af0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc09af90 con 0x562f45e68af0 2024-09-12T02:59:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 10 client.16065 fetch_fsmap learned FSMap version 31 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f45bac620 con 0x562f45e68af0 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbf0005020 con 0x562f45e68af0 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdbfc06a150 con 0x562f45e68af0 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 10 client.16065 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 20 client.16065 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45e85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 4 client.16065 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f45ddaa20 con 0x562f45e83b70 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.863+0000 7fdc04c55640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45e85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.864+0000 7fdc04c55640 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45e85fa0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d26b6383c94e2d server_cookie=bad0b77b4e3c0904 in_seq=0 out_seq=0 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.864+0000 7fdbe8ff9640 10 client.16065 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/4141400792 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f45ddaa20 con 0x562f45e83b70 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdbe8ff9640 10 client.16065 handle_command_reply: tid=0 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 client.16065 shutdown 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 msgr2=0x562f45e70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc09b5c0 tx=0x7fdbfc066700 comp rx=0 tx=0).stop 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 shutdown_connections 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45e85fa0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdbe8ff9640 0 client.16065 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e73cb0 0x562f45e71580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/4141400792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e68af0 0x562f45e70b00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 >> 172.21.15.46:0/4141400792 conn(0x562f45e65060 msgr2=0x562f45e6adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 shutdown_connections 2024-09-12T02:59:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 1 -- 172.21.15.46:0/4141400792 wait complete. 2024-09-12T02:59:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.865+0000 7fdc0a45a740 20 client.16065 trim_cache size 0 max 16384 2024-09-12T02:59:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 Processor -- start 2024-09-12T02:59:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 -- start start 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e8b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001b3a40 con 0x562f45e69410 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001b12b0 con 0x562f45e73cb0 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001af120 con 0x562f45e68af0 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43216/0 (socket says 172.21.15.46:43216) 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 -- 172.21.15.46:0/3245083629 learned_addr learned my addr 172.21.15.46:0/3245083629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc05c57640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e8b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc05456640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 -- 172.21.15.46:0/3245083629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 msgr2=0x562f45e71030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e71030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 -- 172.21.15.46:0/3245083629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 msgr2=0x562f45e8b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e8b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc04c55640 1 -- 172.21.15.46:0/3245083629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f45e64e90 con 0x562f45e69410 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.881+0000 7fdc05456640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e71030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.882+0000 7fdc04c55640 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x562f45e6b690 tx=0x7fdbf00103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc6240b9fbb0d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.882+0000 7fdbeaffd640 1 -- 172.21.15.46:0/3245083629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00052c0 con 0x562f45e69410 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.882+0000 7fdbeaffd640 1 -- 172.21.15.46:0/3245083629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbf0005460 con 0x562f45e69410 2024-09-12T02:59:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.882+0000 7fdbeaffd640 1 -- 172.21.15.46:0/3245083629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00286a0 con 0x562f45e69410 2024-09-12T02:59:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3245083629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 msgr2=0x562f45e71570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x562f45e6b690 tx=0x7fdbf00103d0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3245083629 shutdown_connections 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e73cb0 0x562f45e8b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e69410 0x562f45e71570 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/3245083629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e71030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3245083629 >> 172.21.15.46:0/3245083629 conn(0x562f45e65060 msgr2=0x562f45e72b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3245083629 shutdown_connections 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- 172.21.15.46:0/3245083629 wait complete. 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- start start 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 Processor -- start 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001af770 con 0x562f45e73cb0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001b0be0 con 0x562f45e69410 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001afe40 con 0x562f45e68af0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.883+0000 7fdc0a45a740 10 client.0 _ll_register_callbacks cb 0x7fdbfc0cc750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43222/0 (socket says 172.21.15.46:43222) 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 -- 172.21.15.46:0/2186033953 learned_addr learned my addr 172.21.15.46:0/2186033953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc04c55640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05456640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 -- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 msgr2=0x562f45e7c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e7c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 -- 172.21.15.46:0/2186033953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 msgr2=0x562f45e74b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e74b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 -- 172.21.15.46:0/2186033953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f45e65cf0 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc05c57640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009ca0 tx=0x7fdbf802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9f05bcd1ee885 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8030e20 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fdbf804c420 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.884+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf0005460 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbf804c620 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 10 client.16071 fetch_fsmap learned FSMap version 31 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f45e7aad0 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbf0005460 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdbf8057970 con 0x562f45e73cb0 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 20 client.16071 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45ef0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 4 client.16071 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f45dd4d70 con 0x562f45e83b70 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.885+0000 7fdc05456640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45ef0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.886+0000 7fdc05456640 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45ef0dd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b18541b083b239 server_cookie=9f82515acb4f8477 in_seq=0 out_seq=0 2024-09-12T02:59:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.886+0000 7fdbe8ff9640 10 client.16071 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdbe8ff9640 1 -- 172.21.15.46:0/2186033953 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f45dd4d70 con 0x562f45e83b70 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdbe8ff9640 10 client.16071 handle_command_reply: tid=0 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 client.16071 shutdown 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 msgr2=0x562f45e75070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009ca0 tx=0x7fdbf802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 shutdown_connections 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x562f45e83b70 0x562f45ef0dd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f45e73cb0 0x562f45e75070 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562f45e69410 0x562f45e74b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdbe8ff9640 0 client.16071 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 --2- 172.21.15.46:0/2186033953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f45e68af0 0x562f45e7c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 >> 172.21.15.46:0/2186033953 conn(0x562f45e65060 msgr2=0x562f45e6a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 shutdown_connections 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 1 -- 172.21.15.46:0/2186033953 wait complete. 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.895+0000 7fdc0a45a740 20 client.16071 trim_cache size 0 max 16384 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:59:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 msgr2=0x7fdbec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 0x7fdbec0a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8005e30 tx=0x7fdbf80093d0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 msgr2=0x7fdc0019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c890 tx=0x7fdbfc00cd50 comp rx=0 tx=0).stop 2024-09-12T02:59:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 shutdown_connections 2024-09-12T02:59:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdbec0a2da0 0x7fdbec0a5260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc0010c0d0 0x7fdc0019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc001035c0 0x7fdc0019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 --2- 172.21.15.46:0/1307430290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc00102ca0 0x7fdc0019a3e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 >> 172.21.15.46:0/1307430290 conn(0x7fdc000fc6b0 msgr2=0x7fdc000fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 shutdown_connections 2024-09-12T02:59:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:12.896+0000 7fdc076ba640 1 -- 172.21.15.46:0/1307430290 wait complete. 2024-09-12T02:59:14.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:14 smithi046 ceph-mon[30951]: pgmap v280: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:14 smithi145 ceph-mon[34674]: pgmap v280: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:14 smithi196 ceph-mon[34587]: pgmap v280: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:16.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:16 smithi046 ceph-mon[30951]: pgmap v281: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:16 smithi145 ceph-mon[34674]: pgmap v281: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:16 smithi196 ceph-mon[34587]: pgmap v281: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 18 op/s 2024-09-12T02:59:18.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:18 smithi046 ceph-mon[30951]: pgmap v282: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:18 smithi145 ceph-mon[34674]: pgmap v282: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:18 smithi196 ceph-mon[34587]: pgmap v282: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:20 smithi046 ceph-mon[30951]: pgmap v283: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:20.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:20 smithi145 ceph-mon[34674]: pgmap v283: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:20 smithi196 ceph-mon[34587]: pgmap v283: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-12T02:59:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:21 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:21.921 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:21 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:21.922 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:21 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:22.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:22 smithi046 ceph-mon[30951]: pgmap v284: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:59:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:22 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:22 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:22 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:59:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:22 smithi145 ceph-mon[34674]: pgmap v284: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:59:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:22 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:22 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:22 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:59:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:22 smithi196 ceph-mon[34587]: pgmap v284: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-12T02:59:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:22 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:22 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:22 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T02:59:24.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:24 smithi046 ceph-mon[30951]: pgmap v285: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:24 smithi145 ceph-mon[34674]: pgmap v285: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:24 smithi196 ceph-mon[34587]: pgmap v285: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 -- 172.21.15.46:0/284856719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c101350 msgr2=0x7fb80c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 --2- 172.21.15.46:0/284856719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c101350 0x7fb80c10d880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb804009cd0 tx=0x7fb80402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 -- 172.21.15.46:0/284856719 shutdown_connections 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 --2- 172.21.15.46:0/284856719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c101350 0x7fb80c10d880 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 --2- 172.21.15.46:0/284856719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c1009b0 0x7fb80c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 --2- 172.21.15.46:0/284856719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1069c0 0x7fb80c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 -- 172.21.15.46:0/284856719 >> 172.21.15.46:0/284856719 conn(0x7fb80c077990 msgr2=0x7fb80c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 -- 172.21.15.46:0/284856719 shutdown_connections 2024-09-12T02:59:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.988+0000 7fb812ef1640 1 -- 172.21.15.46:0/284856719 wait complete. 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 Processor -- start 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 -- start start 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1009b0 0x7fb80c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c101350 0x7fb80c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110d00 con 0x7fb80c1069c0 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110b80 con 0x7fb80c101350 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb812ef1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110e80 con 0x7fb80c1009b0 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb81148e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb81148e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60068/0 (socket says 172.21.15.46:60068) 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb81148e640 1 -- 172.21.15.46:0/3330804172 learned_addr learned my addr 172.21.15.46:0/3330804172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c101350 0x7fb80c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.989+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1009b0 0x7fb80c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 -- 172.21.15.46:0/3330804172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1009b0 msgr2=0x7fb80c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1009b0 0x7fb80c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 -- 172.21.15.46:0/3330804172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c101350 msgr2=0x7fb80c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c101350 0x7fb80c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 -- 172.21.15.46:0/3330804172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c1a4650 con 0x7fb80c1069c0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb81148e640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb804005e30 tx=0x7fb804003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb809ffb640 1 -- 172.21.15.46:0/3330804172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804004240 con 0x7fb80c1069c0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb809ffb640 1 -- 172.21.15.46:0/3330804172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb804031ce0 con 0x7fb80c1069c0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c1a4820 con 0x7fb80c1069c0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb809ffb640 1 -- 172.21.15.46:0/3330804172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804046480 con 0x7fb80c1069c0 2024-09-12T02:59:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.990+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a4d30 con 0x7fb80c1069c0 2024-09-12T02:59:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb809ffb640 1 -- 172.21.15.46:0/3330804172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb8040043e0 con 0x7fb80c1069c0 2024-09-12T02:59:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 Processor -- start 2024-09-12T02:59:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 -- start start 2024-09-12T02:59:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c10af0 0x563939c10ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c11410 0x563939c1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563939b752f0 con 0x563939c1bcb0 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563939b1d270 con 0x563939c10af0 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.993+0000 7fb815c92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563939b712b0 con 0x563939c11410 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60074/0 (socket says 172.21.15.46:60074) 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 -- 172.21.15.46:0/206350601 learned_addr learned my addr 172.21.15.46:0/206350601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb810c8d640 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c11410 0x563939c1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c10af0 0x563939c10ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 -- 172.21.15.46:0/206350601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c11410 msgr2=0x563939c1b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c11410 0x563939c1b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 -- 172.21.15.46:0/206350601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c10af0 msgr2=0x563939c10ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c10af0 0x563939c10ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 -- 172.21.15.46:0/206350601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563939b1c340 con 0x563939c1bcb0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb809ffb640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 0x7fb7fc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 0x7fb7fc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb80bfff640 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8011510 tx=0x7fb7f800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e0098339a16ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb809ffb640 1 -- 172.21.15.46:0/3330804172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fb80402e030 con 0x7fb80c1069c0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.994+0000 7fb7f6ffd640 1 -- 172.21.15.46:0/206350601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8005220 con 0x563939c1bcb0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.995+0000 7fb7f6ffd640 1 -- 172.21.15.46:0/206350601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7f8004c50 con 0x563939c1bcb0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.995+0000 7fb7f6ffd640 1 -- 172.21.15.46:0/206350601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800fb20 con 0x563939c1bcb0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.995+0000 7fb815c92740 1 -- 172.21.15.46:0/206350601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 msgr2=0x563939c1e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.995+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 0x7fb7fc0a52e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x563939c0dec0 tx=0x7fb80000b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.995+0000 7fb815c92740 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8011510 tx=0x7fb7f800a750 comp rx=0 tx=0).stop 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- 172.21.15.46:0/206350601 shutdown_connections 2024-09-12T02:59:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c1bcb0 0x563939c1e0a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c11410 0x563939c1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- 172.21.15.46:0/206350601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c10af0 0x563939c10ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- 172.21.15.46:0/206350601 >> 172.21.15.46:0/206350601 conn(0x563939c0d060 msgr2=0x563939c0d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- 172.21.15.46:0/206350601 shutdown_connections 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- 172.21.15.46:0/206350601 wait complete. 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- start start 2024-09-12T02:59:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 Processor -- start 2024-09-12T02:59:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c11410 0x563939c19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c1bcb0 0x563939c19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563939c1e880 con 0x563939c10af0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563939afe290 con 0x563939c11410 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.996+0000 7fb815c92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563939c1ea00 con 0x563939c1bcb0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb815c92740 10 client.0 _ll_register_callbacks cb 0x563939b2f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60080/0 (socket says 172.21.15.46:60080) 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 -- 172.21.15.46:0/3335677432 learned_addr learned my addr 172.21.15.46:0/3335677432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c11410 0x563939c19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb81148e640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c1bcb0 0x563939c19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 -- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c1bcb0 msgr2=0x563939c19680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c1bcb0 0x563939c19680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 -- 172.21.15.46:0/3335677432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c11410 msgr2=0x563939c19140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c11410 0x563939c19140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 -- 172.21.15.46:0/3335677432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563939888210 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb800003f30 tx=0x7fb8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2396295e20d0bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80001b660 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fb80001b800 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.997+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f8004c50 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb80001ba00 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 10 client.16083 fetch_fsmap learned FSMap version 31 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563939954620 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7f8004c50 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb800024070 con 0x563939c10af0 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 10 client.16083 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:59:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 20 client.16083 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c2bcd0 0x563939c2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 4 client.16083 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563939b82a20 con 0x563939c2bcd0 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.998+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c2bcd0 0x563939c2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.999+0000 7fb7ebfff640 10 client.16083 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.999+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c2bcd0 0x563939c2e100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dcb74e179ba9bd server_cookie=d73697d1257b844e in_seq=0 out_seq=0 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.999+0000 7fb7ebfff640 1 -- 172.21.15.46:0/3335677432 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563939b82a20 con 0x563939c2bcd0 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.999+0000 7fb7ebfff640 10 client.16083 handle_command_reply: tid=0 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:25.999+0000 7fb815c92740 1 client.16083 shutdown 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 msgr2=0x563939c18c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb800003f30 tx=0x7fb8000046c0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 shutdown_connections 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c2bcd0 0x563939c2e100 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c1bcb0 0x563939c19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c11410 0x563939c19140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 --2- 172.21.15.46:0/3335677432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c10af0 0x563939c18c00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 >> 172.21.15.46:0/3335677432 conn(0x563939c0d060 msgr2=0x563939c12d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb7ebfff640 0 client.16083 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 shutdown_connections 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 1 -- 172.21.15.46:0/3335677432 wait complete. 2024-09-12T02:59:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.000+0000 7fb815c92740 20 client.16083 trim_cache size 0 max 16384 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 Processor -- start 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 -- start start 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1b16f0 con 0x563939c11410 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1aef60 con 0x563939c1bcb0 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb815c92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1acdd0 con 0x563939c10af0 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.018+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60090/0 (socket says 172.21.15.46:60090) 2024-09-12T02:59:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 -- 172.21.15.46:0/3699340699 learned_addr learned my addr 172.21.15.46:0/3699340699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb810c8d640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 -- 172.21.15.46:0/3699340699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 msgr2=0x563939c194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb81148e640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 -- 172.21.15.46:0/3699340699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 msgr2=0x563939c33fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c33fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 -- 172.21.15.46:0/3699340699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563939c0dcf0 con 0x563939c11410 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb81148e640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c33fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb80bfff640 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80057c0 tx=0x7fb7f800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cab7730e790e300 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb7f67fc640 1 -- 172.21.15.46:0/3699340699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f802b070 con 0x563939c11410 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb7f67fc640 1 -- 172.21.15.46:0/3699340699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7f800f920 con 0x563939c11410 2024-09-12T02:59:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.019+0000 7fb7f67fc640 1 -- 172.21.15.46:0/3699340699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8004a80 con 0x563939c11410 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- 172.21.15.46:0/3699340699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 msgr2=0x563939c33a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80057c0 tx=0x7fb7f800ae90 comp rx=0 tx=0).stop 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- 172.21.15.46:0/3699340699 shutdown_connections 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c33fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c33a60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 --2- 172.21.15.46:0/3699340699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- 172.21.15.46:0/3699340699 >> 172.21.15.46:0/3699340699 conn(0x563939c0d060 msgr2=0x563939c1ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- 172.21.15.46:0/3699340699 shutdown_connections 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- 172.21.15.46:0/3699340699 wait complete. 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.020+0000 7fb815c92740 1 -- start start 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 Processor -- start 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ad420 con 0x563939c11410 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ae890 con 0x563939c1bcb0 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1adaf0 con 0x563939c10af0 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb815c92740 10 client.0 _ll_register_callbacks cb 0x7fb800003bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60104/0 (socket says 172.21.15.46:60104) 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 -- 172.21.15.46:0/1047665313 learned_addr learned my addr 172.21.15.46:0/1047665313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb810c8d640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb81148e640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 -- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 msgr2=0x563939c24320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c24320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 -- 172.21.15.46:0/1047665313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 msgr2=0x563939c15e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c15e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.021+0000 7fb80bfff640 1 -- 172.21.15.46:0/1047665313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563939c165c0 con 0x563939c11410 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb810c8d640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb80bfff640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80058c0 tx=0x7fb7f800a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517fe08e7ee12ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f802b040 con 0x563939c11410 2024-09-12T02:59:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7fb7f800fe10 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f800f920 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7f8044050 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb815c92740 10 client.16089 fetch_fsmap learned FSMap version 31 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563939c96980 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.022+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7f800f920 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb7f801b030 con 0x563939c11410 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 10 client.16089 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 20 client.16089 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c96e80 0x563939c992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 4 client.16089 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563939b7cd70 con 0x563939c96e80 2024-09-12T02:59:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb810c8d640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c96e80 0x563939c992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb810c8d640 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c96e80 0x563939c992a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff0c99a92c6113a server_cookie=f83e175887e8d9da in_seq=0 out_seq=0 2024-09-12T02:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.023+0000 7fb7ebfff640 10 client.16089 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb7ebfff640 1 -- 172.21.15.46:0/1047665313 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563939b7cd70 con 0x563939c96e80 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb7ebfff640 10 client.16089 handle_command_reply: tid=0 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 client.16089 shutdown 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 msgr2=0x563939c158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80058c0 tx=0x7fb7f800a3d0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 shutdown_connections 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x563939c96e80 0x563939c992a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563939c1bcb0 0x563939c15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb7ebfff640 0 client.16089 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563939c11410 0x563939c158f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 --2- 172.21.15.46:0/1047665313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563939c10af0 0x563939c24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 >> 172.21.15.46:0/1047665313 conn(0x563939c0d060 msgr2=0x563939c0b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 shutdown_connections 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 1 -- 172.21.15.46:0/1047665313 wait complete. 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.038+0000 7fb815c92740 20 client.16089 trim_cache size 0 max 16384 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:59:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 msgr2=0x7fb7fc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 0x7fb7fc0a52e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x563939c0dec0 tx=0x7fb80000b390 comp rx=0 tx=0).stop 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 msgr2=0x7fb80c1a4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb804005e30 tx=0x7fb804003ba0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 shutdown_connections 2024-09-12T02:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb7fc0a2e20 0x7fb7fc0a52e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb80c1069c0 0x7fb80c1a4110 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb80c101350 0x7fb80c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 --2- 172.21.15.46:0/3330804172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1009b0 0x7fb80c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 >> 172.21.15.46:0/3330804172 conn(0x7fb80c077990 msgr2=0x7fb80c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 shutdown_connections 2024-09-12T02:59:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:26.039+0000 7fb812ef1640 1 -- 172.21.15.46:0/3330804172 wait complete. 2024-09-12T02:59:26.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:26 smithi196 ceph-mon[34587]: pgmap v286: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:59:26.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:26 smithi046 ceph-mon[30951]: pgmap v286: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:59:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:26 smithi145 ceph-mon[34674]: pgmap v286: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-09-12T02:59:28.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:28 smithi046 ceph-mon[30951]: pgmap v287: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:59:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:28 smithi145 ceph-mon[34674]: pgmap v287: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:59:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:28 smithi196 ceph-mon[34587]: pgmap v287: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-12T02:59:30.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:30 smithi046 ceph-mon[30951]: pgmap v288: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:30 smithi145 ceph-mon[34674]: pgmap v288: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:30 smithi196 ceph-mon[34587]: pgmap v288: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:32.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:32 smithi046 ceph-mon[30951]: pgmap v289: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:32 smithi145 ceph-mon[34674]: pgmap v289: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:32 smithi196 ceph-mon[34587]: pgmap v289: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-12T02:59:34.144 INFO:tasks.workunit.client.0.smithi046.stdout:fs setup took 487 secs 2024-09-12T02:59:34.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:34 smithi046 ceph-mon[30951]: pgmap v290: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:34 smithi145 ceph-mon[34674]: pgmap v290: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:34 smithi196 ceph-mon[34587]: pgmap v290: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-09-12T02:59:35.021 INFO:tasks.workunit.client.0.smithi046.stdout:Syncing()...0 sec 2024-09-12T02:59:35.022 INFO:tasks.workunit.client.0.smithi046.stdout:Starting Actual Benchmark At: Thu Sep 12 02:59:35 2024 2024-09-12T02:59:35.022 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T02:59:36.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:36 smithi046 ceph-mon[30951]: pgmap v291: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:36 smithi145 ceph-mon[34674]: pgmap v291: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:36 smithi196 ceph-mon[34587]: pgmap v291: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 18 op/s 2024-09-12T02:59:38.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:38 smithi046 ceph-mon[30951]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 460 op/s 2024-09-12T02:59:38.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:38 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:38 smithi145 ceph-mon[34674]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 460 op/s 2024-09-12T02:59:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:38 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:38 smithi196 ceph-mon[34587]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 460 op/s 2024-09-12T02:59:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:38 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.338+0000 7f68c10ae640 1 -- 172.21.15.46:0/4172805279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 msgr2=0x7f68bc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.338+0000 7f68c10ae640 1 --2- 172.21.15.46:0/4172805279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 0x7f68bc11d530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009900 tx=0x7f68ac02a3f0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.340+0000 7f68c10ae640 1 -- 172.21.15.46:0/4172805279 shutdown_connections 2024-09-12T02:59:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.340+0000 7f68c10ae640 1 --2- 172.21.15.46:0/4172805279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 0x7f68bc11d530 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.340+0000 7f68c10ae640 1 --2- 172.21.15.46:0/4172805279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.340+0000 7f68c10ae640 1 --2- 172.21.15.46:0/4172805279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 0x7f68bc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.340+0000 7f68c10ae640 1 -- 172.21.15.46:0/4172805279 >> 172.21.15.46:0/4172805279 conn(0x7f68bc06cc60 msgr2=0x7f68bc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.342+0000 7f68c10ae640 1 -- 172.21.15.46:0/4172805279 shutdown_connections 2024-09-12T02:59:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.342+0000 7f68c10ae640 1 -- 172.21.15.46:0/4172805279 wait complete. 2024-09-12T02:59:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.343+0000 7f68c10ae640 1 Processor -- start 2024-09-12T02:59:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.345+0000 7f68c10ae640 1 -- start start 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 0x7f68bc1a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 0x7f68bc1ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1209c0 con 0x7f68bc11b140 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc120840 con 0x7f68bc1091f0 2024-09-12T02:59:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68c10ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc120b40 con 0x7f68bc1095d0 2024-09-12T02:59:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 0x7f68bc1a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:33222/0 (socket says 172.21.15.46:33222) 2024-09-12T02:59:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.346+0000 7f68ba59c640 1 -- 172.21.15.46:0/491057897 learned_addr learned my addr 172.21.15.46:0/491057897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 -- 172.21.15.46:0/491057897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 msgr2=0x7f68bc1a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 0x7f68bc1a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 -- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 msgr2=0x7f68bc1ab790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 0x7f68bc1ab790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 -- 172.21.15.46:0/491057897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc1abec0 con 0x7f68bc1095d0 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.347+0000 7f68ba59c640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68b400cce0 tx=0x7f68b400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.348+0000 7f68a3fff640 1 -- 172.21.15.46:0/491057897 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400f650 con 0x7f68bc1095d0 2024-09-12T02:59:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.348+0000 7f68a3fff640 1 -- 172.21.15.46:0/491057897 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b400fca0 con 0x7f68bc1095d0 2024-09-12T02:59:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.349+0000 7f68a3fff640 1 -- 172.21.15.46:0/491057897 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400eb10 con 0x7f68bc1095d0 2024-09-12T02:59:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.349+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc1ac1b0 con 0x7f68bc1095d0 2024-09-12T02:59:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.350+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1b3a40 con 0x7f68bc1095d0 2024-09-12T02:59:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.354+0000 7f68c3e49740 1 Processor -- start 2024-09-12T02:59:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.354+0000 7f68c3e49740 1 -- start start 2024-09-12T02:59:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.354+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.354+0000 7f68a3fff640 1 -- 172.21.15.46:0/491057897 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f68b400ecb0 con 0x7f68bc1095d0 2024-09-12T02:59:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.354+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97b5e0 0x5566fe985940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe988270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68c3e49740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5566fe8df2f0 con 0x5566fe97b5e0 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68c3e49740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5566fe887270 con 0x5566fe97acc0 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68c3e49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566fe8db2b0 con 0x5566fe985e80 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50530/0 (socket says 172.21.15.46:50530) 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 -- 172.21.15.46:0/1551878356 learned_addr learned my addr 172.21.15.46:0/1551878356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bad9d640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97b5e0 0x5566fe985940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe988270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 -- 172.21.15.46:0/1551878356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 msgr2=0x5566fe988270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe988270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 -- 172.21.15.46:0/1551878356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97b5e0 msgr2=0x5566fe985940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97b5e0 0x5566fe985940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68bb59e640 1 -- 172.21.15.46:0/1551878356 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fe886340 con 0x5566fe97acc0 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.355+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe988270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02a960 tx=0x7f68ac005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f4ed206e27fb52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68a3fff640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 0x7f68a40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68a17fa640 1 -- 172.21.15.46:0/1551878356 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac03e030 con 0x5566fe97acc0 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68a17fa640 1 -- 172.21.15.46:0/1551878356 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac0035f0 con 0x5566fe97acc0 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68bad9d640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 0x7f68a40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.356+0000 7f68a3fff640 1 -- 172.21.15.46:0/491057897 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f68b40cf460 con 0x7f68bc1095d0 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68a17fa640 1 -- 172.21.15.46:0/1551878356 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac035040 con 0x5566fe97acc0 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- 172.21.15.46:0/1551878356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 msgr2=0x5566fe97b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68bad9d640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 0x7f68a40a51a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5566fe978090 tx=0x7f68b0005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02a960 tx=0x7f68ac005780 comp rx=0 tx=0).stop 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- 172.21.15.46:0/1551878356 shutdown_connections 2024-09-12T02:59:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe988270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97b5e0 0x5566fe985940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1551878356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97acc0 0x5566fe97b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- 172.21.15.46:0/1551878356 >> 172.21.15.46:0/1551878356 conn(0x5566fe977230 msgr2=0x5566fe977610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- 172.21.15.46:0/1551878356 shutdown_connections 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- 172.21.15.46:0/1551878356 wait complete. 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.357+0000 7f68c3e49740 1 -- start start 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 Processor -- start 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe982d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe983250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe983790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5566fe988af0 con 0x5566fe97acc0 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe982d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe982d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48452/0 (socket says 172.21.15.46:48452) 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68bad9d640 1 -- 172.21.15.46:0/1844256048 learned_addr learned my addr 172.21.15.46:0/1844256048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5566fe868290 con 0x5566fe97b5e0 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566fe988c70 con 0x5566fe985e80 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe983250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68c3e49740 10 client.0 _ll_register_callbacks cb 0x5566fe899140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.358+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe983790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 -- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 msgr2=0x5566fe983790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe983790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 -- 172.21.15.46:0/1844256048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 msgr2=0x5566fe982d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe982d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fe5f2210 con 0x5566fe97b5e0 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe983790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe983250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc800 tx=0x7f68b40978f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3dddba865b4754 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f689eff5640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40b4060 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f68b40afb70 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac0035f0 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.359+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68b40bb050 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 10 client.25661 fetch_fsmap learned FSMap version 32 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566fe6be620 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac0035f0 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f68b409aaa0 con 0x5566fe97b5e0 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 10 client.25661 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 20 client.25661 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fe9981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 4 client.25661 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566fe8eca20 con 0x5566fe995dc0 2024-09-12T02:59:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.360+0000 7f68bad9d640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fe9981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.361+0000 7f68bad9d640 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fe9981d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e79d75da06d65ebb server_cookie=5419c030fccbfe07 in_seq=0 out_seq=0 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.361+0000 7f689eff5640 10 client.25661 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.362+0000 7f689eff5640 1 -- 172.21.15.46:0/1844256048 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566fe8eca20 con 0x5566fe995dc0 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.362+0000 7f689eff5640 10 client.25661 handle_command_reply: tid=0 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.362+0000 7f68c3e49740 1 client.25661 shutdown 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.363+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 msgr2=0x5566fe983250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.363+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe983250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc800 tx=0x7f68b40978f0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.365+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 shutdown_connections 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fe9981d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f689eff5640 0 client.25661 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe983790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe983250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1844256048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe982d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 >> 172.21.15.46:0/1844256048 conn(0x5566fe977230 msgr2=0x5566fe97cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 shutdown_connections 2024-09-12T02:59:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.366+0000 7f68c3e49740 1 -- 172.21.15.46:0/1844256048 wait complete. 2024-09-12T02:59:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.368+0000 7f68c3e49740 20 client.25661 trim_cache size 0 max 16384 2024-09-12T02:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.407+0000 7f68c3e49740 1 Processor -- start 2024-09-12T02:59:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.407+0000 7f68c3e49740 1 -- start start 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.407+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe971ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.407+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe9835c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe97ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68c3e49740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1c0080 con 0x5566fe97acc0 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68c3e49740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1bd8f0 con 0x5566fe97b5e0 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68c3e49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1bb760 con 0x5566fe985e80 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe9835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe9835c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50554/0 (socket says 172.21.15.46:50554) 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68ba59c640 1 -- 172.21.15.46:0/3920331757 learned_addr learned my addr 172.21.15.46:0/3920331757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68bb59e640 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe97ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.408+0000 7f68bad9d640 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe971ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.409+0000 7f68bad9d640 1 -- 172.21.15.46:0/3920331757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 msgr2=0x5566fe97ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.409+0000 7f68bad9d640 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe97ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.409+0000 7f68bad9d640 1 -- 172.21.15.46:0/3920331757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 msgr2=0x5566fe9835c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.409+0000 7f68bad9d640 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe9835c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.409+0000 7f68bad9d640 1 -- 172.21.15.46:0/3920331757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fe994980 con 0x5566fe97acc0 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.410+0000 7f68bad9d640 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe971ed0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5566fe998a20 tx=0x7f68b001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715842a56d9fb0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.411+0000 7f68a0ff9640 1 -- 172.21.15.46:0/3920331757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b001fd10 con 0x5566fe97acc0 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.411+0000 7f68a0ff9640 1 -- 172.21.15.46:0/3920331757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b00054d0 con 0x5566fe97acc0 2024-09-12T02:59:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.411+0000 7f68a0ff9640 1 -- 172.21.15.46:0/3920331757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b002b6f0 con 0x5566fe97acc0 2024-09-12T02:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 -- 172.21.15.46:0/3920331757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 msgr2=0x5566fe971ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe971ed0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5566fe998a20 tx=0x7f68b001b000 comp rx=0 tx=0).stop 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 -- 172.21.15.46:0/3920331757 shutdown_connections 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe985e80 0x5566fe97ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe97b5e0 0x5566fe9835c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 --2- 172.21.15.46:0/3920331757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe971ed0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 -- 172.21.15.46:0/3920331757 >> 172.21.15.46:0/3920331757 conn(0x5566fe977230 msgr2=0x5566fe97c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 -- 172.21.15.46:0/3920331757 shutdown_connections 2024-09-12T02:59:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.412+0000 7f68c3e49740 1 -- 172.21.15.46:0/3920331757 wait complete. 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 -- start start 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 Processor -- start 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe98e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe985e80 0x5566fe97ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1bbdb0 con 0x5566fe97acc0 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1bd220 con 0x5566fe985e80 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.413+0000 7f68c3e49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1bc480 con 0x5566fe97b5e0 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68c3e49740 10 client.0 _ll_register_callbacks cb 0x7f68b400e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:33266/0 (socket says 172.21.15.46:33266) 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 -- 172.21.15.46:0/1685810194 learned_addr learned my addr 172.21.15.46:0/1685810194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68bb59e640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe985e80 0x5566fe97ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 -- 172.21.15.46:0/1685810194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe985e80 msgr2=0x5566fe97ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe985e80 0x5566fe97ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 -- 172.21.15.46:0/1685810194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 msgr2=0x5566fe98e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe98e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.414+0000 7f68ba59c640 1 -- 172.21.15.46:0/1685810194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fe980700 con 0x5566fe97b5e0 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.415+0000 7f68ba59c640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ff90 tx=0x7f68b40a43b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc753085b1e29d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.415+0000 7f689eff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T02:59:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.421+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b409afb0 con 0x5566fe97b5e0 2024-09-12T02:59:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.422+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b00054d0 con 0x5566fe97b5e0 2024-09-12T02:59:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.422+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f68b40d1dd0 con 0x5566fe97b5e0 2024-09-12T02:59:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.422+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68b40a8020 con 0x5566fe97b5e0 2024-09-12T02:59:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.423+0000 7f68c3e49740 10 client.25627 fetch_fsmap learned FSMap version 32 2024-09-12T02:59:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.423+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566fea00a70 con 0x5566fe97b5e0 2024-09-12T02:59:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.424+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b00054d0 con 0x5566fe97b5e0 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.424+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f68b409a510 con 0x5566fe97b5e0 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 10 client.25627 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 20 client.25627 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fea03240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 4 client.25627 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:59:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566fe8e6d70 con 0x5566fe995dc0 2024-09-12T02:59:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.425+0000 7f68bad9d640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fea03240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.429+0000 7f68bad9d640 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fea03240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8502dafe68e59b66 server_cookie=daf72dbcc7de60ba in_seq=0 out_seq=0 2024-09-12T02:59:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.430+0000 7f689eff5640 10 client.25627 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.454+0000 7f689eff5640 1 -- 172.21.15.46:0/1685810194 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566fe8e6d70 con 0x5566fe995dc0 2024-09-12T02:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.454+0000 7f689eff5640 10 client.25627 handle_command_reply: tid=0 2024-09-12T02:59:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.454+0000 7f68c3e49740 1 client.25627 shutdown 2024-09-12T02:59:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.456+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 msgr2=0x5566fe97f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.456+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ff90 tx=0x7f68b40a43b0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 shutdown_connections 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5566fe995dc0 0x5566fea03240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5566fe985e80 0x5566fe97ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566fe97b5e0 0x5566fe97f9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 --2- 172.21.15.46:0/1685810194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566fe97acc0 0x5566fe98e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 >> 172.21.15.46:0/1685810194 conn(0x5566fe977230 msgr2=0x5566fe976260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.457+0000 7f689eff5640 0 client.25627 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.458+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 shutdown_connections 2024-09-12T02:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.458+0000 7f68c3e49740 1 -- 172.21.15.46:0/1685810194 wait complete. 2024-09-12T02:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.458+0000 7f68c3e49740 20 client.25627 trim_cache size 0 max 16384 2024-09-12T02:59:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:59:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 msgr2=0x7f68a40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 0x7f68a40a51a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5566fe978090 tx=0x7f68b0005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 msgr2=0x7f68bc1a7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68b400cce0 tx=0x7f68b400a620 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 shutdown_connections 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f68a40a2ce0 0x7f68a40a51a0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5566fe978090 tx=0x7f68b0005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc11b140 0x7f68bc1ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68bc1095d0 0x7f68bc1a7410 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68b400cce0 tx=0x7f68b400a620 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 --2- 172.21.15.46:0/491057897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68bc1091f0 0x7f68bc1a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.460+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 >> 172.21.15.46:0/491057897 conn(0x7f68bc06cc60 msgr2=0x7f68bc11b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.461+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 shutdown_connections 2024-09-12T02:59:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:39.461+0000 7f68c10ae640 1 -- 172.21.15.46:0/491057897 wait complete. 2024-09-12T02:59:40.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:40 smithi046 ceph-mon[30951]: pgmap v293: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:40.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:40 smithi145 ceph-mon[34674]: pgmap v293: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:40 smithi196 ceph-mon[34587]: pgmap v293: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:42.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:42 smithi046 ceph-mon[30951]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:42.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:42 smithi145 ceph-mon[34674]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:42 smithi196 ceph-mon[34587]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 449 op/s 2024-09-12T02:59:43.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:43 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:43 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:43.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:43 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:59:44.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:44 smithi145 ceph-mon[34674]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 2.41k op/s 2024-09-12T02:59:44.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:44 smithi196 ceph-mon[34587]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 2.41k op/s 2024-09-12T02:59:44.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:44 smithi046 ceph-mon[30951]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 2.41k op/s 2024-09-12T02:59:46.584 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:46 smithi046 ceph-mon[30951]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 2.40k op/s 2024-09-12T02:59:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:46 smithi145 ceph-mon[34674]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 2.40k op/s 2024-09-12T02:59:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:46 smithi196 ceph-mon[34587]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 2.40k op/s 2024-09-12T02:59:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:48 smithi145 ceph-mon[34674]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 4.32k op/s 2024-09-12T02:59:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:48 smithi196 ceph-mon[34587]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 4.32k op/s 2024-09-12T02:59:48.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:48 smithi046 ceph-mon[30951]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 4.32k op/s 2024-09-12T02:59:49.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:49 smithi145 ceph-mon[34674]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:49.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:49 smithi196 ceph-mon[34587]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:49.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:49 smithi046 ceph-mon[30951]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:52.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:52 smithi046 ceph-mon[30951]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:52 smithi145 ceph-mon[34674]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 -- 172.21.15.46:0/970845781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294106800 msgr2=0x7f8294106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 --2- 172.21.15.46:0/970845781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294106800 0x7f8294106be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8284009890 tx=0x7f828402a370 comp rx=0 tx=0).stop 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 -- 172.21.15.46:0/970845781 shutdown_connections 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 --2- 172.21.15.46:0/970845781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8294101190 0x7f829410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 --2- 172.21.15.46:0/970845781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82941007f0 0x7f8294100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 --2- 172.21.15.46:0/970845781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294106800 0x7f8294106be0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8284009890 tx=0x7f828402a370 comp rx=0 tx=0).stop 2024-09-12T02:59:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 -- 172.21.15.46:0/970845781 >> 172.21.15.46:0/970845781 conn(0x7f82940777c0 msgr2=0x7f82940feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.640+0000 7f829b24c640 1 -- 172.21.15.46:0/970845781 shutdown_connections 2024-09-12T02:59:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.643+0000 7f829b24c640 1 -- 172.21.15.46:0/970845781 wait complete. 2024-09-12T02:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.644+0000 7f829b24c640 1 Processor -- start 2024-09-12T02:59:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.644+0000 7f829b24c640 1 -- start start 2024-09-12T02:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82941007f0 0x7f8294198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 0x7f8294198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8294110ab0 con 0x7f829419cc00 2024-09-12T02:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8294110930 con 0x7f82941007f0 2024-09-12T02:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.645+0000 7f829b24c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8294110c30 con 0x7f8294101190 2024-09-12T02:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53968/0 (socket says 172.21.15.46:53968) 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 -- 172.21.15.46:0/4053112760 learned_addr learned my addr 172.21.15.46:0/4053112760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 -- 172.21.15.46:0/4053112760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 msgr2=0x7f8294198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f8293fff640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 0x7f8294198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 0x7f8294198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 -- 172.21.15.46:0/4053112760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82941007f0 msgr2=0x7f8294198340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82941007f0 0x7f8294198340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f82997e9640 1 -- 172.21.15.46:0/4053112760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829419d6f0 con 0x7f829419cc00 2024-09-12T02:59:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.646+0000 7f8293fff640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 0x7f8294198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.647+0000 7f82997e9640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f828800c970 tx=0x7f828800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.647+0000 7f8291ffb640 1 -- 172.21.15.46:0/4053112760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288007bb0 con 0x7f829419cc00 2024-09-12T02:59:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.647+0000 7f8291ffb640 1 -- 172.21.15.46:0/4053112760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8288007d50 con 0x7f829419cc00 2024-09-12T02:59:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.647+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829419d9e0 con 0x7f829419cc00 2024-09-12T02:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.647+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82940fce30 con 0x7f829419cc00 2024-09-12T02:59:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.648+0000 7f8291ffb640 1 -- 172.21.15.46:0/4053112760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288018730 con 0x7f829419cc00 2024-09-12T02:59:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.652+0000 7f8291ffb640 1 -- 172.21.15.46:0/4053112760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f828800f450 con 0x7f829419cc00 2024-09-12T02:59:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.656+0000 7f8291ffb640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 0x7f827c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.656+0000 7f8291ffb640 1 -- 172.21.15.46:0/4053112760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f82880cf730 con 0x7f829419cc00 2024-09-12T02:59:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 Processor -- start 2024-09-12T02:59:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 -- start start 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f8298fe8640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 0x7f827c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf4500 0x555d63d07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 0x555d63d09a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d63c582f0 con 0x555d63d07690 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d63c00270 con 0x555d63cf4500 2024-09-12T02:59:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.658+0000 7f829dfe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555d63c542b0 con 0x555d63cf3be0 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.659+0000 7f8298fe8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.659+0000 7f8298fe8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:42966/0 (socket says 172.21.15.46:42966) 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.659+0000 7f8298fe8640 1 -- 172.21.15.46:0/3326010685 learned_addr learned my addr 172.21.15.46:0/3326010685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f82997e9640 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 0x555d63d09a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 -- 172.21.15.46:0/3326010685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf4500 msgr2=0x555d63d07150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:59:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf4500 0x555d63d07150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 -- 172.21.15.46:0/3326010685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 msgr2=0x555d63d09a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 0x555d63d09a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 -- 172.21.15.46:0/3326010685 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d63bff340 con 0x555d63cf3be0 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f82997e9640 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 0x555d63d09a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f828402a960 tx=0x7f8284042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4d78e47f44abd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.660+0000 7f8298fe8640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 0x7f827c0a52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82840095f0 tx=0x7f8284009540 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.661+0000 7f827affd640 1 -- 172.21.15.46:0/3326010685 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284037070 con 0x555d63cf3be0 2024-09-12T02:59:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.661+0000 7f827affd640 1 -- 172.21.15.46:0/3326010685 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8284032990 con 0x555d63cf3be0 2024-09-12T02:59:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.661+0000 7f827affd640 1 -- 172.21.15.46:0/3326010685 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82840306a0 con 0x555d63cf3be0 2024-09-12T02:59:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.664+0000 7f829dfe6740 1 -- 172.21.15.46:0/3326010685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 msgr2=0x555d63cf3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.664+0000 7f829dfe6740 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f828402a960 tx=0x7f8284042000 comp rx=0 tx=0).stop 2024-09-12T02:59:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.665+0000 7f829dfe6740 1 -- 172.21.15.46:0/3326010685 shutdown_connections 2024-09-12T02:59:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.665+0000 7f829dfe6740 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d07690 0x555d63d09a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.665+0000 7f829dfe6740 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf4500 0x555d63d07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.665+0000 7f829dfe6740 1 --2- 172.21.15.46:0/3326010685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63cf3fc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.665+0000 7f829dfe6740 1 -- 172.21.15.46:0/3326010685 >> 172.21.15.46:0/3326010685 conn(0x555d63cf0230 msgr2=0x555d63cf0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.667+0000 7f829dfe6740 1 -- 172.21.15.46:0/3326010685 shutdown_connections 2024-09-12T02:59:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.668+0000 7f829dfe6740 1 -- 172.21.15.46:0/3326010685 wait complete. 2024-09-12T02:59:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.668+0000 7f829dfe6740 1 -- start start 2024-09-12T02:59:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.669+0000 7f829dfe6740 1 Processor -- start 2024-09-12T02:59:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf3be0 0x555d63d01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 0x555d63d017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d63cf6b80 con 0x555d63cf3be0 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555d63be1290 con 0x555d63d07690 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f82997e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f82997e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:60244/0 (socket says 172.21.15.46:60244) 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f82997e9640 1 -- 172.21.15.46:0/1996018063 learned_addr learned my addr 172.21.15.46:0/1996018063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.670+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555d63cf6d00 con 0x555d63cf4500 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.671+0000 7f8293fff640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 0x555d63d017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.671+0000 7f829dfe6740 10 client.0 _ll_register_callbacks cb 0x555d63c12140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.671+0000 7f8298fe8640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf3be0 0x555d63d01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.672+0000 7f8293fff640 1 -- 172.21.15.46:0/1996018063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 msgr2=0x555d63d06f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.672+0000 7f8293fff640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.672+0000 7f8293fff640 1 -- 172.21.15.46:0/1996018063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf3be0 msgr2=0x555d63d01260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.672+0000 7f8293fff640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf3be0 0x555d63d01260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.672+0000 7f8293fff640 1 -- 172.21.15.46:0/1996018063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d6396b210 con 0x555d63cf4500 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.673+0000 7f82997e9640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.673+0000 7f8293fff640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 0x555d63d017a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82940ffe60 tx=0x7f828c00ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917527797f12f65e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.673+0000 7f8273fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T02:59:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.674+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c00dba0 con 0x555d63cf4500 2024-09-12T02:59:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.674+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f828c00dd40 con 0x555d63cf4500 2024-09-12T02:59:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.676+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8284032990 con 0x555d63cf4500 2024-09-12T02:59:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.678+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f828c004b20 con 0x555d63cf4500 2024-09-12T02:59:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.678+0000 7f829dfe6740 10 client.25636 fetch_fsmap learned FSMap version 33 2024-09-12T02:59:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.678+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d63a37620 con 0x555d63cf4500 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.680+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8284032990 con 0x555d63cf4500 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.680+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f828c00b040 con 0x555d63cf4500 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 10 client.25636 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 20 client.25636 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d1ffb0 0x555d63d223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 4 client.25636 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d63c65a20 con 0x555d63d1ffb0 2024-09-12T02:59:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.681+0000 7f8298fe8640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d1ffb0 0x555d63d223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.683+0000 7f8298fe8640 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d1ffb0 0x555d63d223d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ba5f263e90eaf0 server_cookie=8965b8d344fdbe78 in_seq=0 out_seq=0 2024-09-12T02:59:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.683+0000 7f8273fff640 10 client.25636 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.684+0000 7f8273fff640 1 -- 172.21.15.46:0/1996018063 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d63c65a20 con 0x555d63d1ffb0 2024-09-12T02:59:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.684+0000 7f8273fff640 10 client.25636 handle_command_reply: tid=0 2024-09-12T02:59:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.684+0000 7f829dfe6740 1 client.25636 shutdown 2024-09-12T02:59:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.684+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 msgr2=0x555d63d017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.684+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 0x555d63d017a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82940ffe60 tx=0x7f828c00ff00 comp rx=0 tx=0).stop 2024-09-12T02:59:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 shutdown_connections 2024-09-12T02:59:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d1ffb0 0x555d63d223d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d06f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf4500 0x555d63d017a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1996018063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf3be0 0x555d63d01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 >> 172.21.15.46:0/1996018063 conn(0x555d63cf0230 msgr2=0x555d63cf5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.685+0000 7f8273fff640 0 client.25636 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.686+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 shutdown_connections 2024-09-12T02:59:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.688+0000 7f829dfe6740 1 -- 172.21.15.46:0/1996018063 wait complete. 2024-09-12T02:59:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.688+0000 7f829dfe6740 20 client.25636 trim_cache size 0 max 16384 2024-09-12T02:59:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:52 smithi196 ceph-mon[34587]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T02:59:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.734+0000 7f829dfe6740 1 Processor -- start 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.734+0000 7f829dfe6740 1 -- start start 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 0x555d63d06dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 0x555d63cfb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63d07690 0x555d63d27ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8280012980 con 0x555d63cf4500 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82800101f0 con 0x555d63cf3be0 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f829dfe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f828000e060 con 0x555d63d07690 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f8293fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 0x555d63cfb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f8293fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 0x555d63cfb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53986/0 (socket says 172.21.15.46:53986) 2024-09-12T02:59:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.735+0000 7f8293fff640 1 -- 172.21.15.46:0/46700516 learned_addr learned my addr 172.21.15.46:0/46700516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.736+0000 7f8298fe8640 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 0x555d63d06dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.736+0000 7f82997e9640 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63d07690 0x555d63d27ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 -- 172.21.15.46:0/46700516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63d07690 msgr2=0x555d63d27ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63d07690 0x555d63d27ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 -- 172.21.15.46:0/46700516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 msgr2=0x555d63cfb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 0x555d63cfb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 -- 172.21.15.46:0/46700516 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d63cf0060 con 0x555d63cf3be0 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.737+0000 7f8298fe8640 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 0x555d63d06dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8284000c00 tx=0x7f8284009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26726867287f8d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.739+0000 7f827a7fc640 1 -- 172.21.15.46:0/46700516 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284037040 con 0x555d63cf3be0 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.739+0000 7f827a7fc640 1 -- 172.21.15.46:0/46700516 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8284032990 con 0x555d63cf3be0 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.739+0000 7f827a7fc640 1 -- 172.21.15.46:0/46700516 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284050040 con 0x555d63cf3be0 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 -- 172.21.15.46:0/46700516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 msgr2=0x555d63d06dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 0x555d63d06dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8284000c00 tx=0x7f8284009760 comp rx=0 tx=0).stop 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 -- 172.21.15.46:0/46700516 shutdown_connections 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63d07690 0x555d63d27ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63cf4500 0x555d63cfb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 --2- 172.21.15.46:0/46700516 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63cf3be0 0x555d63d06dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.740+0000 7f829dfe6740 1 -- 172.21.15.46:0/46700516 >> 172.21.15.46:0/46700516 conn(0x555d63cf0230 msgr2=0x555d63cfae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.741+0000 7f829dfe6740 1 -- 172.21.15.46:0/46700516 shutdown_connections 2024-09-12T02:59:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.741+0000 7f829dfe6740 1 -- 172.21.15.46:0/46700516 wait complete. 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 -- start start 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 Processor -- start 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63d2edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f828000e6b0 con 0x555d63d2f9a0 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f828000fb20 con 0x555d63d07690 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f828000ed80 con 0x555d63cf3be0 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.742+0000 7f829dfe6740 10 client.0 _ll_register_callbacks cb 0x7f828c0095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.743+0000 7f8298fe8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63d2edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.744+0000 7f82997e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.744+0000 7f82997e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54002/0 (socket says 172.21.15.46:54002) 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.744+0000 7f82997e9640 1 -- 172.21.15.46:0/1499887154 learned_addr learned my addr 172.21.15.46:0/1499887154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.744+0000 7f8293fff640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 -- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 msgr2=0x555d63d2edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63d2edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 -- 172.21.15.46:0/1499887154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 msgr2=0x555d63d2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 -- 172.21.15.46:0/1499887154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d63d9a3c0 con 0x555d63d2f9a0 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.745+0000 7f82997e9640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f828809b540 tx=0x7f828809fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29732fd86cc84be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.746+0000 7f8273fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T02:59:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.746+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880b9060 con 0x555d63d2f9a0 2024-09-12T02:59:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.746+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9467+0+0 (secure 0 0 0) 0x7f82880cf040 con 0x555d63d2f9a0 2024-09-12T02:59:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.746+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8284032990 con 0x555d63d2f9a0 2024-09-12T02:59:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.747+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f82880b3030 con 0x555d63d2f9a0 2024-09-12T02:59:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.748+0000 7f829dfe6740 10 client.16116 fetch_fsmap learned FSMap version 33 2024-09-12T02:59:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.748+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d63d9b6d0 con 0x555d63d2f9a0 2024-09-12T02:59:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.748+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8284032990 con 0x555d63d2f9a0 2024-09-12T02:59:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.748+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f8288020030 con 0x555d63d2f9a0 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 10 client.16116 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 20 client.16116 populate_metadata read hostname 'smithi046' 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d9bde0 0x555d63d9e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 4 client.16116 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:59:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d63c5fd70 con 0x555d63d9bde0 2024-09-12T02:59:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.749+0000 7f8298fe8640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d9bde0 0x555d63d9e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.750+0000 7f8298fe8640 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d9bde0 0x555d63d9e200 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48559eaeb930be2 server_cookie=8890bf678bcba20b in_seq=0 out_seq=0 2024-09-12T02:59:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.750+0000 7f8273fff640 10 client.16116 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f8273fff640 1 -- 172.21.15.46:0/1499887154 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d63c5fd70 con 0x555d63d9bde0 2024-09-12T02:59:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f8273fff640 10 client.16116 handle_command_reply: tid=0 2024-09-12T02:59:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 client.16116 shutdown 2024-09-12T02:59:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 msgr2=0x555d63d99b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f828809b540 tx=0x7f828809fcb0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 shutdown_connections 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x555d63d9bde0 0x555d63d9e200 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d63d2f9a0 0x555d63d99b90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555d63d07690 0x555d63d2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 --2- 172.21.15.46:0/1499887154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d63cf3be0 0x555d63d2edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.770+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 >> 172.21.15.46:0/1499887154 conn(0x555d63cf0230 msgr2=0x555d63ceea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.772+0000 7f8273fff640 0 client.16116 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T02:59:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.772+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 shutdown_connections 2024-09-12T02:59:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.772+0000 7f829dfe6740 1 -- 172.21.15.46:0/1499887154 wait complete. 2024-09-12T02:59:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.772+0000 7f829dfe6740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 msgr2=0x7f827c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 0x7f827c0a52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82840095f0 tx=0x7f8284009540 comp rx=0 tx=0).stop 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 msgr2=0x7f829419d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f828800c970 tx=0x7f828800ce30 comp rx=0 tx=0).stop 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 shutdown_connections 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f827c0a2df0 0x7f827c0a52b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829419cc00 0x7f829419d0b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8294101190 0x7f8294198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 --2- 172.21.15.46:0/4053112760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82941007f0 0x7f8294198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 >> 172.21.15.46:0/4053112760 conn(0x7f82940777c0 msgr2=0x7f8294103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.774+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 shutdown_connections 2024-09-12T02:59:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T02:59:52.775+0000 7f829b24c640 1 -- 172.21.15.46:0/4053112760 wait complete. 2024-09-12T02:59:54.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:54 smithi046 ceph-mon[30951]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-09-12T02:59:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:54 smithi145 ceph-mon[34674]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-09-12T02:59:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:54 smithi196 ceph-mon[34587]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-09-12T02:59:56.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:56 smithi196 ceph-mon[34587]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T02:59:56.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:56 smithi046 ceph-mon[30951]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T02:59:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:56 smithi145 ceph-mon[34674]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T02:59:58.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 02:59:58 smithi046 ceph-mon[30951]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.81k op/s 2024-09-12T02:59:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 02:59:58 smithi145 ceph-mon[34674]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.81k op/s 2024-09-12T02:59:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 02:59:58 smithi196 ceph-mon[34587]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.81k op/s 2024-09-12T03:00:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:00 smithi046 ceph-mon[30951]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:00.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:00 smithi046 ceph-mon[30951]: overall HEALTH_OK 2024-09-12T03:00:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:00 smithi145 ceph-mon[34674]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:00 smithi145 ceph-mon[34674]: overall HEALTH_OK 2024-09-12T03:00:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:00 smithi196 ceph-mon[34587]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:00 smithi196 ceph-mon[34587]: overall HEALTH_OK 2024-09-12T03:00:02.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:02 smithi046 ceph-mon[30951]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:02.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:02 smithi145 ceph-mon[34674]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:02 smithi196 ceph-mon[34587]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-09-12T03:00:03.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:03 smithi145 ceph-mon[34674]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:00:03.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:03 smithi196 ceph-mon[34587]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:00:03.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:03 smithi046 ceph-mon[30951]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:00:04.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:04 smithi145 ceph-mon[34674]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:04 smithi196 ceph-mon[34587]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:04.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:04 smithi046 ceph-mon[30951]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.009+0000 7f186e650640 1 -- 172.21.15.46:0/3998052887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681091f0 msgr2=0x7f1868110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.009+0000 7f186e650640 1 --2- 172.21.15.46:0/3998052887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681091f0 0x7f1868110250 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ca0 tx=0x7f186002a8f0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.016+0000 7f186e650640 1 -- 172.21.15.46:0/3998052887 shutdown_connections 2024-09-12T03:00:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.016+0000 7f186e650640 1 --2- 172.21.15.46:0/3998052887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f186811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.016+0000 7f186e650640 1 --2- 172.21.15.46:0/3998052887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681095d0 0x7f1868109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.016+0000 7f186e650640 1 --2- 172.21.15.46:0/3998052887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681091f0 0x7f1868110250 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.016+0000 7f186e650640 1 -- 172.21.15.46:0/3998052887 >> 172.21.15.46:0/3998052887 conn(0x7f186806cc60 msgr2=0x7f186806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.018+0000 7f186e650640 1 -- 172.21.15.46:0/3998052887 shutdown_connections 2024-09-12T03:00:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.023+0000 7f186e650640 1 -- 172.21.15.46:0/3998052887 wait complete. 2024-09-12T03:00:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.023+0000 7f186e650640 1 Processor -- start 2024-09-12T03:00:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 -- start start 2024-09-12T03:00:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 0x7f1868118740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 0x7f1868111810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f1868111d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1868120a10 con 0x7f18681095d0 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1868120890 con 0x7f18681091f0 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.024+0000 7f186e650640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1868120b90 con 0x7f186811b140 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.025+0000 7f186cbed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f1868111d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.025+0000 7f186cbed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f1868111d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:58406/0 (socket says 172.21.15.46:58406) 2024-09-12T03:00:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.025+0000 7f186cbed640 1 -- 172.21.15.46:0/3366399243 learned_addr learned my addr 172.21.15.46:0/3366399243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.025+0000 7f1867fff640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 0x7f1868118740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.025+0000 7f18677fe640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 0x7f1868111810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.027+0000 7f1867fff640 1 -- 172.21.15.46:0/3366399243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 msgr2=0x7f1868111d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.027+0000 7f1867fff640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f1868111d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.027+0000 7f1867fff640 1 -- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 msgr2=0x7f1868111810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.027+0000 7f1867fff640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 0x7f1868111810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.027+0000 7f1867fff640 1 -- 172.21.15.46:0/3366399243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1868112340 con 0x7f18681091f0 2024-09-12T03:00:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.028+0000 7f1867fff640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 0x7f1868118740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f186002aee0 tx=0x7f1860003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.028+0000 7f18677fe640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 0x7f1868111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.028+0000 7f18657fa640 1 -- 172.21.15.46:0/3366399243 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860004040 con 0x7f18681091f0 2024-09-12T03:00:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.028+0000 7f18657fa640 1 -- 172.21.15.46:0/3366399243 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f186002d070 con 0x7f18681091f0 2024-09-12T03:00:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.028+0000 7f18657fa640 1 -- 172.21.15.46:0/3366399243 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186002fc90 con 0x7f18681091f0 2024-09-12T03:00:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.030+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681125d0 con 0x7f18681091f0 2024-09-12T03:00:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.030+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1868116550 con 0x7f18681091f0 2024-09-12T03:00:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 Processor -- start 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 -- start start 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 0x564a75b2a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b34eb0 0x564a75b372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564a75a8e2f0 con 0x564a75b29cf0 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564a75a36270 con 0x564a75b2a610 2024-09-12T03:00:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.034+0000 7f18713e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a75a8a2b0 con 0x564a75b34eb0 2024-09-12T03:00:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.037+0000 7f1867fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.037+0000 7f1867fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b34970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:60254/0 (socket says 172.21.15.46:60254) 2024-09-12T03:00:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.037+0000 7f1867fff640 1 -- 172.21.15.46:0/1246808807 learned_addr learned my addr 172.21.15.46:0/1246808807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.037+0000 7f18657fa640 1 -- 172.21.15.46:0/3366399243 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f18600041e0 con 0x7f18681091f0 2024-09-12T03:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.037+0000 7f186cbed640 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 0x564a75b2a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.039+0000 7f18657fa640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 0x7f18500a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.039+0000 7f18657fa640 1 -- 172.21.15.46:0/3366399243 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9555+0+0 (secure 0 0 0) 0x7f18600f0af0 con 0x7f18681091f0 2024-09-12T03:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.039+0000 7f186cbed640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 0x7f18500a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.041+0000 7f186cbed640 1 -- 172.21.15.46:0/1246808807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b34eb0 msgr2=0x564a75b372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.041+0000 7f186cbed640 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b34eb0 0x564a75b372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.041+0000 7f186cbed640 1 -- 172.21.15.46:0/1246808807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 msgr2=0x564a75b34970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.041+0000 7f186cbed640 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b34970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.041+0000 7f186cbed640 1 -- 172.21.15.46:0/1246808807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a75a35340 con 0x564a75b29cf0 2024-09-12T03:00:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.042+0000 7f186cbed640 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 0x564a75b2a0d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f185c0139d0 tx=0x7f185c013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79950d8beb06536d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.043+0000 7f186cbed640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 0x7f18500a53c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f185c0107a0 tx=0x7f185c01e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.043+0000 7f184e7fc640 1 -- 172.21.15.46:0/1246808807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c023070 con 0x564a75b29cf0 2024-09-12T03:00:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.043+0000 7f184e7fc640 1 -- 172.21.15.46:0/1246808807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185c009e20 con 0x564a75b29cf0 2024-09-12T03:00:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.044+0000 7f18713e1740 1 -- 172.21.15.46:0/1246808807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 msgr2=0x564a75b2a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.044+0000 7f18713e1740 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 0x564a75b2a0d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f185c0139d0 tx=0x7f185c013e90 comp rx=0 tx=0).stop 2024-09-12T03:00:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.051+0000 7f18713e1740 1 -- 172.21.15.46:0/1246808807 shutdown_connections 2024-09-12T03:00:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.051+0000 7f18713e1740 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b34eb0 0x564a75b372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.051+0000 7f18713e1740 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b34970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.051+0000 7f18713e1740 1 --2- 172.21.15.46:0/1246808807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b29cf0 0x564a75b2a0d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.051+0000 7f18713e1740 1 -- 172.21.15.46:0/1246808807 >> 172.21.15.46:0/1246808807 conn(0x564a75b26260 msgr2=0x564a75b26640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.063+0000 7f18713e1740 1 -- 172.21.15.46:0/1246808807 shutdown_connections 2024-09-12T03:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.063+0000 7f18713e1740 1 -- 172.21.15.46:0/1246808807 wait complete. 2024-09-12T03:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.064+0000 7f18713e1740 1 -- start start 2024-09-12T03:00:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.066+0000 7f18713e1740 1 Processor -- start 2024-09-12T03:00:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b32220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564a75b37ad0 con 0x564a75b34eb0 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564a75a17290 con 0x564a75b29cf0 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a75b37c50 con 0x564a75b2a610 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.067+0000 7f18713e1740 10 client.0 _ll_register_callbacks cb 0x564a75a48140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:60260/0 (socket says 172.21.15.46:60260) 2024-09-12T03:00:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 -- 172.21.15.46:0/1774063872 learned_addr learned my addr 172.21.15.46:0/1774063872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 -- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 msgr2=0x564a75b32220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:00:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b32220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 -- 172.21.15.46:0/1774063872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 msgr2=0x564a75b32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.068+0000 7f1867fff640 1 -- 172.21.15.46:0/1774063872 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a757a1210 con 0x564a75b29cf0 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.069+0000 7f1867fff640 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18600bcd60 tx=0x7f18600bcba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4b21053f9dcb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.069+0000 7f18477fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.069+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186002f340 con 0x564a75b29cf0 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.069+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c009e20 con 0x564a75b29cf0 2024-09-12T03:00:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.069+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9555+0+0 (secure 0 0 0) 0x7f18600b9440 con 0x564a75b29cf0 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.071+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f186002f640 con 0x564a75b29cf0 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.071+0000 7f18713e1740 10 client.25688 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.071+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a7586d620 con 0x564a75b29cf0 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185c009e20 con 0x564a75b29cf0 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f18600d2020 con 0x564a75b29cf0 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 10 client.25688 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 20 client.25688 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75b44f40 0x564a75b47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 4 client.25688 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.072+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a75a9ba20 con 0x564a75b44f40 2024-09-12T03:00:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.083+0000 7f18677fe640 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75b44f40 0x564a75b47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.085+0000 7f18477fe640 10 client.25688 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.085+0000 7f18677fe640 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75b44f40 0x564a75b47400 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a3e200d4062e1d server_cookie=9ffbac9a8815a574 in_seq=0 out_seq=0 2024-09-12T03:00:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.086+0000 7f18477fe640 1 -- 172.21.15.46:0/1774063872 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a75a9ba20 con 0x564a75b44f40 2024-09-12T03:00:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.086+0000 7f18477fe640 10 client.25688 handle_command_reply: tid=0 2024-09-12T03:00:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.086+0000 7f18713e1740 1 client.25688 shutdown 2024-09-12T03:00:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 msgr2=0x564a75b31ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18600bcd60 tx=0x7f18600bcba0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 shutdown_connections 2024-09-12T03:00:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75b44f40 0x564a75b47400 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b32220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 --2- 172.21.15.46:0/1774063872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b31ce0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 >> 172.21.15.46:0/1774063872 conn(0x564a75b26260 msgr2=0x564a75b2bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18477fe640 0 client.25688 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 shutdown_connections 2024-09-12T03:00:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.101+0000 7f18713e1740 1 -- 172.21.15.46:0/1774063872 wait complete. 2024-09-12T03:00:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.102+0000 7f18713e1740 20 client.25688 trim_cache size 0 max 16384 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 Processor -- start 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 -- start start 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b29cf0 0x564a75b325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b4cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c8d40 con 0x564a75b34eb0 2024-09-12T03:00:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c65b0 con 0x564a75b2a610 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.138+0000 7f18713e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c4420 con 0x564a75b29cf0 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37122/0 (socket says 172.21.15.46:37122) 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 -- 172.21.15.46:0/613041398 learned_addr learned my addr 172.21.15.46:0/613041398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 -- 172.21.15.46:0/613041398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b29cf0 msgr2=0x564a75b325b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b29cf0 0x564a75b325b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 -- 172.21.15.46:0/613041398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 msgr2=0x564a75b4cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b4cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 -- 172.21.15.46:0/613041398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a75b375d0 con 0x564a75b34eb0 2024-09-12T03:00:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.139+0000 7f186cbed640 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f185c009980 tx=0x7f185c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15fa92239360cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.140+0000 7f184dffb640 1 -- 172.21.15.46:0/613041398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c02c020 con 0x564a75b34eb0 2024-09-12T03:00:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.141+0000 7f184dffb640 1 -- 172.21.15.46:0/613041398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185c00ec30 con 0x564a75b34eb0 2024-09-12T03:00:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.141+0000 7f184dffb640 1 -- 172.21.15.46:0/613041398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c023040 con 0x564a75b34eb0 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 -- 172.21.15.46:0/613041398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 msgr2=0x564a75b4d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f185c009980 tx=0x7f185c009ed0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 -- 172.21.15.46:0/613041398 shutdown_connections 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b4d250 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b2a610 0x564a75b4cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 --2- 172.21.15.46:0/613041398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b29cf0 0x564a75b325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 -- 172.21.15.46:0/613041398 >> 172.21.15.46:0/613041398 conn(0x564a75b26260 msgr2=0x564a75b30e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 -- 172.21.15.46:0/613041398 shutdown_connections 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.143+0000 7f18713e1740 1 -- 172.21.15.46:0/613041398 wait complete. 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 -- start start 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 Processor -- start 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b2f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c4a70 con 0x564a75b34eb0 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c5ee0 con 0x564a75b29cf0 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18681c5140 con 0x564a75b2a610 2024-09-12T03:00:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.144+0000 7f18713e1740 10 client.0 _ll_register_callbacks cb 0x7f18600f8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.145+0000 7f18677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.145+0000 7f18677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:58418/0 (socket says 172.21.15.46:58418) 2024-09-12T03:00:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.145+0000 7f18677fe640 1 -- 172.21.15.46:0/1070800585 learned_addr learned my addr 172.21.15.46:0/1070800585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.146+0000 7f1867fff640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f18677fe640 1 -- 172.21.15.46:0/1070800585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 msgr2=0x564a75b3d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f18677fe640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b3d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f186cbed640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b2f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f18677fe640 1 -- 172.21.15.46:0/1070800585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 msgr2=0x564a75b2f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f18677fe640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b2f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.147+0000 7f18677fe640 1 -- 172.21.15.46:0/1070800585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a75b2f880 con 0x564a75b2a610 2024-09-12T03:00:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.148+0000 7f186cbed640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b2f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.148+0000 7f18677fe640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1858005d80 tx=0x7f1858011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690bcc8ee7180255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.148+0000 7f18477fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:00:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185801c070 con 0x564a75b2a610 2024-09-12T03:00:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9555+0+0 (secure 0 0 0) 0x7f1858005220 con 0x564a75b2a610 2024-09-12T03:00:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c00ec30 con 0x564a75b2a610 2024-09-12T03:00:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1858025050 con 0x564a75b2a610 2024-09-12T03:00:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18713e1740 10 client.25651 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.150+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a75bafe20 con 0x564a75b2a610 2024-09-12T03:00:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.151+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185c00ec30 con 0x564a75b2a610 2024-09-12T03:00:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f185800f040 con 0x564a75b2a610 2024-09-12T03:00:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 10 client.25651 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 20 client.25651 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75bb0530 0x564a75bb2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 4 client.25651 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.152+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a75a95d70 con 0x564a75bb0530 2024-09-12T03:00:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.153+0000 7f1867fff640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75bb0530 0x564a75bb2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.156+0000 7f1867fff640 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75bb0530 0x564a75bb2950 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74812cd6c74abae6 server_cookie=c27fa3a609a697fb in_seq=0 out_seq=0 2024-09-12T03:00:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.156+0000 7f18477fe640 10 client.25651 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:06 smithi145 ceph-mon[34674]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.650+0000 7f18477fe640 1 -- 172.21.15.46:0/1070800585 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a75a95d70 con 0x564a75bb0530 2024-09-12T03:00:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.650+0000 7f18477fe640 10 client.25651 handle_command_reply: tid=0 2024-09-12T03:00:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.650+0000 7f18713e1740 1 client.25651 shutdown 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 msgr2=0x564a75b2ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1858005d80 tx=0x7f1858011810 comp rx=0 tx=0).stop 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 shutdown_connections 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564a75bb0530 0x564a75bb2950 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a75b34eb0 0x564a75b2f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a75b2a610 0x564a75b2ead0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 --2- 172.21.15.46:0/1070800585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564a75b29cf0 0x564a75b3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 >> 172.21.15.46:0/1070800585 conn(0x564a75b26260 msgr2=0x564a75b2b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18477fe640 0 client.25651 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 shutdown_connections 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.654+0000 7f18713e1740 1 -- 172.21.15.46:0/1070800585 wait complete. 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.655+0000 7f18713e1740 20 client.25651 trim_cache size 0 max 16384 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:00:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:00:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:00:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.659+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 msgr2=0x7f18500a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.659+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 0x7f18500a53c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f185c0107a0 tx=0x7f185c01e000 comp rx=0 tx=0).stop 2024-09-12T03:00:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.659+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 msgr2=0x7f1868118740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.659+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 0x7f1868118740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f186002aee0 tx=0x7f1860003ba0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 shutdown_connections 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f18500a2f00 0x7f18500a53c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186811b140 0x7f1868111d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18681095d0 0x7f1868111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 --2- 172.21.15.46:0/3366399243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18681091f0 0x7f1868118740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.662+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 >> 172.21.15.46:0/3366399243 conn(0x7f186806cc60 msgr2=0x7f186811b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.663+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 shutdown_connections 2024-09-12T03:00:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:06.663+0000 7f186e650640 1 -- 172.21.15.46:0/3366399243 wait complete. 2024-09-12T03:00:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:06 smithi196 ceph-mon[34587]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:06.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:06 smithi046 ceph-mon[30951]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:08 smithi145 ceph-mon[34674]: pgmap v308: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:00:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:08 smithi196 ceph-mon[34587]: pgmap v308: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:00:08.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:08 smithi046 ceph-mon[30951]: pgmap v308: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:00:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:10 smithi145 ceph-mon[34674]: pgmap v309: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2024-09-12T03:00:10.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:10 smithi145 ceph-mon[34674]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:00:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:10 smithi196 ceph-mon[34587]: pgmap v309: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2024-09-12T03:00:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:10 smithi196 ceph-mon[34587]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:00:10.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:10 smithi046 ceph-mon[30951]: pgmap v309: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2024-09-12T03:00:10.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:10 smithi046 ceph-mon[30951]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:00:12.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:12 smithi145 ceph-mon[34674]: pgmap v311: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-12T03:00:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:12 smithi196 ceph-mon[34587]: pgmap v311: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-12T03:00:12.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:12 smithi046 ceph-mon[30951]: pgmap v311: 129 pgs: 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 123 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-09-12T03:00:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:14 smithi145 ceph-mon[34674]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:14.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:14 smithi196 ceph-mon[34587]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:14 smithi046 ceph-mon[30951]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:16 smithi046 ceph-mon[30951]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:16.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:16 smithi046 ceph-mon[30951]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:00:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:16 smithi145 ceph-mon[34674]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:16 smithi145 ceph-mon[34674]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:00:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:16 smithi196 ceph-mon[34587]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-09-12T03:00:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:16 smithi196 ceph-mon[34587]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:00:18.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:18 smithi046 ceph-mon[30951]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-09-12T03:00:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:18 smithi145 ceph-mon[34674]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-09-12T03:00:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:18 smithi196 ceph-mon[34587]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.918+0000 7f89994e9640 1 -- 172.21.15.46:0/570684972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994103af0 msgr2=0x7f8994103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.918+0000 7f89994e9640 1 --2- 172.21.15.46:0/570684972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994103af0 0x7f8994103f50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f898801a940 tx=0x7f898803b470 comp rx=0 tx=0).stop 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.920+0000 7f89994e9640 1 -- 172.21.15.46:0/570684972 shutdown_connections 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.920+0000 7f89994e9640 1 --2- 172.21.15.46:0/570684972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994104490 0x7f89941109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.920+0000 7f89994e9640 1 --2- 172.21.15.46:0/570684972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994103af0 0x7f8994103f50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.920+0000 7f89994e9640 1 --2- 172.21.15.46:0/570684972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994109b90 0x7f8994109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.920+0000 7f89994e9640 1 -- 172.21.15.46:0/570684972 >> 172.21.15.46:0/570684972 conn(0x7f89940ff770 msgr2=0x7f8994101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.925+0000 7f89994e9640 1 -- 172.21.15.46:0/570684972 shutdown_connections 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.926+0000 7f89994e9640 1 -- 172.21.15.46:0/570684972 wait complete. 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 Processor -- start 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 -- start start 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994103af0 0x7f89941105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994104490 0x7f899410c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f899406e4a0 con 0x7f8994109b90 2024-09-12T03:00:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f899406e320 con 0x7f8994104490 2024-09-12T03:00:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.927+0000 7f89994e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f899406e620 con 0x7f8994103af0 2024-09-12T03:00:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.928+0000 7f8998ce8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.928+0000 7f8998ce8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42526/0 (socket says 172.21.15.46:42526) 2024-09-12T03:00:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.928+0000 7f8998ce8640 1 -- 172.21.15.46:0/777167695 learned_addr learned my addr 172.21.15.46:0/777167695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.929+0000 7f89937fe640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994104490 0x7f899410c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8993fff640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994103af0 0x7f89941105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8998ce8640 1 -- 172.21.15.46:0/777167695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994103af0 msgr2=0x7f89941105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8998ce8640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994103af0 0x7f89941105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8998ce8640 1 -- 172.21.15.46:0/777167695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994104490 msgr2=0x7f899410c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8998ce8640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994104490 0x7f899410c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.930+0000 7f8998ce8640 1 -- 172.21.15.46:0/777167695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899410d340 con 0x7f8994109b90 2024-09-12T03:00:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.932+0000 7f8998ce8640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b990 tx=0x7f898c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.942+0000 7f89917fa640 1 -- 172.21.15.46:0/777167695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c019070 con 0x7f8994109b90 2024-09-12T03:00:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.942+0000 7f89917fa640 1 -- 172.21.15.46:0/777167695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f898c00cba0 con 0x7f8994109b90 2024-09-12T03:00:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.942+0000 7f89917fa640 1 -- 172.21.15.46:0/777167695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c007590 con 0x7f8994109b90 2024-09-12T03:00:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.945+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899419e310 con 0x7f8994109b90 2024-09-12T03:00:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.945+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899419e6c0 con 0x7f8994109b90 2024-09-12T03:00:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.948+0000 7f899d5c5740 1 Processor -- start 2024-09-12T03:00:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 -- start start 2024-09-12T03:00:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c381df0 0x55618c38c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55618c2dd2f0 con 0x55618c38c690 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55618c285270 con 0x55618c381df0 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f899d5c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55618c2d92b0 con 0x55618c3814d0 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f89917fa640 1 -- 172.21.15.46:0/777167695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f898c014440 con 0x7f8994109b90 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f89937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f8993fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c381df0 0x55618c38c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.951+0000 7f89937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:43268/0 (socket says 172.21.15.46:43268) 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 -- 172.21.15.46:0/1224902135 learned_addr learned my addr 172.21.15.46:0/1224902135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 -- 172.21.15.46:0/1224902135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c381df0 msgr2=0x55618c38c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c381df0 0x55618c38c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 -- 172.21.15.46:0/1224902135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 msgr2=0x55618c38ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 -- 172.21.15.46:0/1224902135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618c284340 con 0x55618c3814d0 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.952+0000 7f89937fe640 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f898803ba60 tx=0x7f898801a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4390cf9ca5c1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.953+0000 7f89917fa640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 0x7f897c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.954+0000 7f89917fa640 1 -- 172.21.15.46:0/777167695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f898c0cec00 con 0x7f8994109b90 2024-09-12T03:00:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.955+0000 7f8993fff640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 0x7f897c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.957+0000 7f8993fff640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 0x7f897c0a53c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55618c37e8a0 tx=0x7f8984005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.962+0000 7f897a7fc640 1 -- 172.21.15.46:0/1224902135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988046030 con 0x55618c3814d0 2024-09-12T03:00:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.962+0000 7f897a7fc640 1 -- 172.21.15.46:0/1224902135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89880049b0 con 0x55618c3814d0 2024-09-12T03:00:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.962+0000 7f897a7fc640 1 -- 172.21.15.46:0/1224902135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988040910 con 0x55618c3814d0 2024-09-12T03:00:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.963+0000 7f899d5c5740 1 -- 172.21.15.46:0/1224902135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 msgr2=0x55618c3818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.963+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f898803ba60 tx=0x7f898801a5f0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 -- 172.21.15.46:0/1224902135 shutdown_connections 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c381df0 0x55618c38c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1224902135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c3814d0 0x55618c3818b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 -- 172.21.15.46:0/1224902135 >> 172.21.15.46:0/1224902135 conn(0x55618c37da40 msgr2=0x55618c37de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 -- 172.21.15.46:0/1224902135 shutdown_connections 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 -- 172.21.15.46:0/1224902135 wait complete. 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.966+0000 7f899d5c5740 1 -- start start 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 Processor -- start 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c381df0 0x55618c389bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55618c38f260 con 0x55618c381df0 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55618c266290 con 0x55618c3814d0 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55618c38f3e0 con 0x55618c38c690 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.967+0000 7f899d5c5740 10 client.0 _ll_register_callbacks cb 0x55618c297140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.968+0000 7f8998ce8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.968+0000 7f8998ce8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:43272/0 (socket says 172.21.15.46:43272) 2024-09-12T03:00:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.968+0000 7f8998ce8640 1 -- 172.21.15.46:0/1852952924 learned_addr learned my addr 172.21.15.46:0/1852952924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.968+0000 7f8993fff640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 -- 172.21.15.46:0/1852952924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 msgr2=0x55618c389670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 -- 172.21.15.46:0/1852952924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c381df0 msgr2=0x55618c389bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c381df0 0x55618c389bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 -- 172.21.15.46:0/1852952924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618bff0210 con 0x55618c38c690 2024-09-12T03:00:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.969+0000 7f8998ce8640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b960 tx=0x7f898c014ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1804665231a225a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.981+0000 7f89737fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:00:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.982+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0a3ac0 con 0x55618c38c690 2024-09-12T03:00:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.982+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f898c0974c0 con 0x55618c38c690 2024-09-12T03:00:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.982+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89880049b0 con 0x55618c38c690 2024-09-12T03:00:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.986+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f898c0cc1f0 con 0x55618c38c690 2024-09-12T03:00:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.986+0000 7f899d5c5740 10 client.25660 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.986+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55618c0bc620 con 0x55618c38c690 2024-09-12T03:00:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.987+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89880049b0 con 0x55618c38c690 2024-09-12T03:00:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f898c096ba0 con 0x55618c38c690 2024-09-12T03:00:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 10 client.25660 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 20 client.25660 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c378bd0 0x55618c37b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 4 client.25660 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.988+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618c2eaa20 con 0x55618c378bd0 2024-09-12T03:00:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:19.997+0000 7f8993fff640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c378bd0 0x55618c37b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.006+0000 7f8993fff640 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c378bd0 0x55618c37b090 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9176cde4d7b613f3 server_cookie=925c73d11dffcf16 in_seq=0 out_seq=0 2024-09-12T03:00:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.009+0000 7f89737fe640 10 client.25660 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.009+0000 7f89737fe640 1 -- 172.21.15.46:0/1852952924 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55618c2eaa20 con 0x55618c378bd0 2024-09-12T03:00:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.009+0000 7f89737fe640 10 client.25660 handle_command_reply: tid=0 2024-09-12T03:00:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.009+0000 7f899d5c5740 1 client.25660 shutdown 2024-09-12T03:00:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 msgr2=0x55618c38a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b960 tx=0x7f898c014ac0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 shutdown_connections 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c378bd0 0x55618c37b090 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c38c690 0x55618c38a0f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c381df0 0x55618c389bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 --2- 172.21.15.46:0/1852952924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.012+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 >> 172.21.15.46:0/1852952924 conn(0x55618c37da40 msgr2=0x55618c3837a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.014+0000 7f89737fe640 0 client.25660 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.014+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 shutdown_connections 2024-09-12T03:00:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.014+0000 7f899d5c5740 1 -- 172.21.15.46:0/1852952924 wait complete. 2024-09-12T03:00:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.014+0000 7f899d5c5740 20 client.25660 trim_cache size 0 max 16384 2024-09-12T03:00:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.052+0000 7f899d5c5740 1 Processor -- start 2024-09-12T03:00:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.052+0000 7f899d5c5740 1 -- start start 2024-09-12T03:00:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.052+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.052+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c37c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f8993fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f8993fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:49306/0 (socket says 172.21.15.46:49306) 2024-09-12T03:00:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f8993fff640 1 -- 172.21.15.46:0/3673152602 learned_addr learned my addr 172.21.15.46:0/3673152602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f899d5c5740 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c37ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f89937fe640 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c37c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a69d0 con 0x55618c38c690 2024-09-12T03:00:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a4240 con 0x55618c3814d0 2024-09-12T03:00:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.053+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a2140 con 0x55618c381df0 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.054+0000 7f8998ce8640 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c37ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 -- 172.21.15.46:0/3673152602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 msgr2=0x55618c37c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c37c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 -- 172.21.15.46:0/3673152602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 msgr2=0x55618c37ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c37ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 -- 172.21.15.46:0/3673152602 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618c37b670 con 0x55618c3814d0 2024-09-12T03:00:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.055+0000 7f8993fff640 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8984005b10 tx=0x7f898401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3175a1e0fdd7a412 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.058+0000 7f8979ffb640 1 -- 172.21.15.46:0/3673152602 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840037d0 con 0x55618c3814d0 2024-09-12T03:00:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.058+0000 7f8979ffb640 1 -- 172.21.15.46:0/3673152602 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8984003e20 con 0x55618c3814d0 2024-09-12T03:00:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.058+0000 7f8979ffb640 1 -- 172.21.15.46:0/3673152602 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984021410 con 0x55618c3814d0 2024-09-12T03:00:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.066+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 msgr2=0x55618c389d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.066+0000 7f899d5c5740 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8984005b10 tx=0x7f898401a000 comp rx=0 tx=0).stop 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.068+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 shutdown_connections 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.068+0000 7f899d5c5740 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c37ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.068+0000 7f899d5c5740 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c37c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.068+0000 7f899d5c5740 1 --2- 172.21.15.46:0/3673152602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c389d40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.068+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 >> 172.21.15.46:0/3673152602 conn(0x55618c37da40 msgr2=0x55618c389630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.069+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 shutdown_connections 2024-09-12T03:00:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.070+0000 7f899d5c5740 1 -- 172.21.15.46:0/3673152602 wait complete. 2024-09-12T03:00:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.070+0000 7f899d5c5740 1 -- start start 2024-09-12T03:00:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.071+0000 7f899d5c5740 1 Processor -- start 2024-09-12T03:00:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.071+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c3946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f89937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f89937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:43294/0 (socket says 172.21.15.46:43294) 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f89937fe640 1 -- 172.21.15.46:0/4084771158 learned_addr learned my addr 172.21.15.46:0/4084771158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a2700 con 0x55618c38c690 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a3b70 con 0x55618c3814d0 2024-09-12T03:00:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a2dd0 con 0x55618c381df0 2024-09-12T03:00:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.072+0000 7f899d5c5740 10 client.0 _ll_register_callbacks cb 0x7f898c014c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.074+0000 7f89937fe640 1 -- 172.21.15.46:0/4084771158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 msgr2=0x55618c3946b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:00:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.074+0000 7f89937fe640 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c3946b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.074+0000 7f89937fe640 1 -- 172.21.15.46:0/4084771158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 msgr2=0x55618c38f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.074+0000 7f89937fe640 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.074+0000 7f89937fe640 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618c38fba0 con 0x55618c381df0 2024-09-12T03:00:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.075+0000 7f89937fe640 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55618c388ee0 tx=0x7f8988004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa315c56f487f1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.075+0000 7f89737fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:00:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.075+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988060020 con 0x55618c381df0 2024-09-12T03:00:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.075+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f8988005130 con 0x55618c381df0 2024-09-12T03:00:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.076+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8984003e20 con 0x55618c381df0 2024-09-12T03:00:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.078+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8988046070 con 0x55618c381df0 2024-09-12T03:00:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.083+0000 7f899d5c5740 10 client.25666 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.084+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55618c40ffa0 con 0x55618c381df0 2024-09-12T03:00:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.084+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8988040510 con 0x55618c381df0 2024-09-12T03:00:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.085+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f898804b040 con 0x55618c381df0 2024-09-12T03:00:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 10 client.25666 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 20 client.25666 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c410290 0x55618c412680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 4 client.25666 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:00:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.088+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8984003e20 con 0x55618c410290 2024-09-12T03:00:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.091+0000 7f8993fff640 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c410290 0x55618c412680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.097+0000 7f8993fff640 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c410290 0x55618c412680 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36742ec731bc733 server_cookie=d9afd1cbe53178c0 in_seq=0 out_seq=0 2024-09-12T03:00:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.097+0000 7f89737fe640 10 client.25666 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.177+0000 7f89737fe640 1 -- 172.21.15.46:0/4084771158 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8984003e20 con 0x55618c410290 2024-09-12T03:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.177+0000 7f89737fe640 10 client.25666 handle_command_reply: tid=0 2024-09-12T03:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.177+0000 7f899d5c5740 1 client.25666 shutdown 2024-09-12T03:00:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.183+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 msgr2=0x55618c394bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.183+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55618c388ee0 tx=0x7f8988004770 comp rx=0 tx=0).stop 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 shutdown_connections 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55618c410290 0x55618c412680 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55618c38c690 0x55618c38f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55618c381df0 0x55618c394bf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 --2- 172.21.15.46:0/4084771158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55618c3814d0 0x55618c3946b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 >> 172.21.15.46:0/4084771158 conn(0x55618c37da40 msgr2=0x55618c373a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f89737fe640 0 client.25666 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 shutdown_connections 2024-09-12T03:00:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.186+0000 7f899d5c5740 1 -- 172.21.15.46:0/4084771158 wait complete. 2024-09-12T03:00:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.187+0000 7f899d5c5740 20 client.25666 trim_cache size 0 max 16384 2024-09-12T03:00:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:00:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:00:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:00:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:00:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.200+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 msgr2=0x7f897c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.200+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 0x7f897c0a53c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55618c37e8a0 tx=0x7f8984005f90 comp rx=0 tx=0).stop 2024-09-12T03:00:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.200+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 msgr2=0x7f899410cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.200+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b990 tx=0x7f898c00be50 comp rx=0 tx=0).stop 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.201+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 shutdown_connections 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.201+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f897c0a2f00 0x7f897c0a53c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.201+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8994109b90 0x7f899410cc10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.201+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8994104490 0x7f899410c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.202+0000 7f89994e9640 1 --2- 172.21.15.46:0/777167695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8994103af0 0x7f89941105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.202+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 >> 172.21.15.46:0/777167695 conn(0x7f89940ff770 msgr2=0x7f89940fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.205+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 shutdown_connections 2024-09-12T03:00:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:20.205+0000 7f89994e9640 1 -- 172.21.15.46:0/777167695 wait complete. 2024-09-12T03:00:20.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:20 smithi046 ceph-mon[30951]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-09-12T03:00:20.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:20 smithi145 ceph-mon[34674]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-09-12T03:00:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:20 smithi196 ceph-mon[34587]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-09-12T03:00:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:22 smithi046 ceph-mon[30951]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:00:22.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:22 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:22.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:22 smithi145 ceph-mon[34674]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:00:22.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:22 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:22 smithi196 ceph-mon[34587]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:00:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:22 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: pgmap v318: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:24 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: pgmap v318: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:24 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: pgmap v318: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:24.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:24 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:25 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T03:00:25.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:25 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:00:25.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:25 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T03:00:25.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:25 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:00:25.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:25 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi145 to 176.0M 2024-09-12T03:00:25.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:25 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi145 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:00:26.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:26 smithi145 ceph-mon[34674]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:26 smithi196 ceph-mon[34587]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:26.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:26 smithi046 ceph-mon[30951]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:00:28.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:28 smithi145 ceph-mon[34674]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2024-09-12T03:00:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:28 smithi196 ceph-mon[34587]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2024-09-12T03:00:28.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:28 smithi046 ceph-mon[30951]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2024-09-12T03:00:30.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:30 smithi145 ceph-mon[34674]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:30 smithi196 ceph-mon[34587]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:30.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:30 smithi046 ceph-mon[30951]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:31 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:31 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:31.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:31 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:32 smithi145 ceph-mon[34674]: pgmap v322: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:00:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:32 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T03:00:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:32 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:00:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:32 smithi196 ceph-mon[34587]: pgmap v322: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:00:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:32 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T03:00:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:32 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:00:32.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:32 smithi046 ceph-mon[30951]: pgmap v322: 129 pgs: 129 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:00:32.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:32 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-12T03:00:32.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:32 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi196 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 -- 172.21.15.46:0/3383006421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0158109460 msgr2=0x7f0158109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 --2- 172.21.15.46:0/3383006421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0158109460 0x7f0158109840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f014c00af00 tx=0x7f014c00ac00 comp rx=0 tx=0).stop 2024-09-12T03:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 -- 172.21.15.46:0/3383006421 shutdown_connections 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 --2- 172.21.15.46:0/3383006421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158112920 0x7f0158114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 --2- 172.21.15.46:0/3383006421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158109e10 0x7f0158112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 --2- 172.21.15.46:0/3383006421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0158109460 0x7f0158109840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 -- 172.21.15.46:0/3383006421 >> 172.21.15.46:0/3383006421 conn(0x7f015806d040 msgr2=0x7f015806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 -- 172.21.15.46:0/3383006421 shutdown_connections 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.430+0000 7f015c8e7640 1 -- 172.21.15.46:0/3383006421 wait complete. 2024-09-12T03:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.431+0000 7f015c8e7640 1 Processor -- start 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.431+0000 7f015c8e7640 1 -- start start 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158112920 0x7f01581a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f015819e590 0x7f015819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01581181c0 con 0x7f0158109460 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0158118040 con 0x7f015819e590 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f015c8e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0158118340 con 0x7f0158112920 2024-09-12T03:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39538/0 (socket says 172.21.15.46:39538) 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 -- 172.21.15.46:0/2302728123 learned_addr learned my addr 172.21.15.46:0/2302728123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f0157fff640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f015819e590 0x7f015819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f0156ffd640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158112920 0x7f01581a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 -- 172.21.15.46:0/2302728123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158112920 msgr2=0x7f01581a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158112920 0x7f01581a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 -- 172.21.15.46:0/2302728123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f015819e590 msgr2=0x7f015819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f015819e590 0x7f015819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.432+0000 7f01577fe640 1 -- 172.21.15.46:0/2302728123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015819f0b0 con 0x7f0158109460 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f01577fe640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f014c004770 tx=0x7f014c00a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f0154ff9640 1 -- 172.21.15.46:0/2302728123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c048070 con 0x7f0158109460 2024-09-12T03:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f0154ff9640 1 -- 172.21.15.46:0/2302728123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c004b60 con 0x7f0158109460 2024-09-12T03:00:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015819f340 con 0x7f0158109460 2024-09-12T03:00:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f0154ff9640 1 -- 172.21.15.46:0/2302728123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0035f0 con 0x7f0158109460 2024-09-12T03:00:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.433+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581ab590 con 0x7f0158109460 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f0154ff9640 1 -- 172.21.15.46:0/2302728123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f014c04b070 con 0x7f0158109460 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 Processor -- start 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 -- start start 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb3ffcc0 0x5555eb4000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb40d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555eb3642f0 con 0x5555eb4005e0 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5555eb30c270 con 0x5555eb40ae80 2024-09-12T03:00:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01609bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5555eb3602b0 con 0x5555eb3ffcc0 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f0154ff9640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 0x7f01400a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39548/0 (socket says 172.21.15.46:39548) 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f01577fe640 1 -- 172.21.15.46:0/803659801 learned_addr learned my addr 172.21.15.46:0/803659801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.436+0000 7f0156ffd640 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb40d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f0157fff640 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb3ffcc0 0x5555eb4000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f0157fff640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 0x7f01400a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f0154ff9640 1 -- 172.21.15.46:0/2302728123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f014c0f7ff0 con 0x7f0158109460 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 -- 172.21.15.46:0/803659801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb3ffcc0 msgr2=0x5555eb4000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb3ffcc0 0x5555eb4000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 -- 172.21.15.46:0/803659801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 msgr2=0x5555eb40d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb40d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 -- 172.21.15.46:0/803659801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eb30b340 con 0x5555eb4005e0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f01577fe640 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f014c0f52c0 tx=0x7f014c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d9bad7ef90cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f013dffb640 1 -- 172.21.15.46:0/803659801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c052030 con 0x5555eb4005e0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f0157fff640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 0x7f01400a5200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f015000bfd0 tx=0x7f015000d040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f013dffb640 1 -- 172.21.15.46:0/803659801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0c4850 con 0x5555eb4005e0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.437+0000 7f013dffb640 1 -- 172.21.15.46:0/803659801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0fa110 con 0x5555eb4005e0 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- 172.21.15.46:0/803659801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 msgr2=0x5555eb40a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f014c0f52c0 tx=0x7f014c00c040 comp rx=0 tx=0).stop 2024-09-12T03:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- 172.21.15.46:0/803659801 shutdown_connections 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb40d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40a940 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 --2- 172.21.15.46:0/803659801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb3ffcc0 0x5555eb4000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- 172.21.15.46:0/803659801 >> 172.21.15.46:0/803659801 conn(0x5555eb3fc230 msgr2=0x5555eb3fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- 172.21.15.46:0/803659801 shutdown_connections 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- 172.21.15.46:0/803659801 wait complete. 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.438+0000 7f01609bb740 1 -- start start 2024-09-12T03:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 Processor -- start 2024-09-12T03:00:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb4083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb408920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555eb40d810 con 0x5555eb3ffcc0 2024-09-12T03:00:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5555eb2ed290 con 0x5555eb40ae80 2024-09-12T03:00:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5555eb40d990 con 0x5555eb4005e0 2024-09-12T03:00:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01609bb740 10 client.0 _ll_register_callbacks cb 0x5555eb31e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39550/0 (socket says 172.21.15.46:39550) 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 -- 172.21.15.46:0/999044693 learned_addr learned my addr 172.21.15.46:0/999044693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f0156ffd640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb4083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 -- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 msgr2=0x5555eb4083e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb4083e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 -- 172.21.15.46:0/999044693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 msgr2=0x5555eb408920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb408920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 -- 172.21.15.46:0/999044693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eb077210 con 0x5555eb3ffcc0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f01577fe640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c05b0 tx=0x7f014c0e4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575368206c661b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.439+0000 7f0136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c052070 con 0x5555eb3ffcc0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f014c0f5600 con 0x5555eb3ffcc0 2024-09-12T03:00:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014c0c4850 con 0x5555eb3ffcc0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f014c0c4150 con 0x5555eb3ffcc0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 10 client.16146 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555eb143620 con 0x5555eb3ffcc0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0c4850 con 0x5555eb3ffcc0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f014c0bffc0 con 0x5555eb3ffcc0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 10 client.16146 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 20 client.16146 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb41ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 4 client.16146 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555eb371a20 con 0x5555eb41aa40 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.440+0000 7f0156ffd640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb41ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.441+0000 7f0136ffd640 10 client.16146 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.441+0000 7f0156ffd640 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb41ce30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be139b6cfa84fe server_cookie=67e482d6f70389f6 in_seq=0 out_seq=0 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.441+0000 7f0136ffd640 1 -- 172.21.15.46:0/999044693 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555eb371a20 con 0x5555eb41aa40 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.441+0000 7f0136ffd640 10 client.16146 handle_command_reply: tid=0 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.441+0000 7f01609bb740 1 client.16146 shutdown 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 msgr2=0x5555eb407ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c05b0 tx=0x7f014c0e4300 comp rx=0 tx=0).stop 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 shutdown_connections 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb41ce30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f0136ffd640 0 client.16146 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb408920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb4083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 --2- 172.21.15.46:0/999044693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb407ea0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 >> 172.21.15.46:0/999044693 conn(0x5555eb3fc230 msgr2=0x5555eb401f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 shutdown_connections 2024-09-12T03:00:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 1 -- 172.21.15.46:0/999044693 wait complete. 2024-09-12T03:00:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.442+0000 7f01609bb740 20 client.16146 trim_cache size 0 max 16384 2024-09-12T03:00:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.456+0000 7f01609bb740 1 Processor -- start 2024-09-12T03:00:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.456+0000 7f01609bb740 1 -- start start 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb3fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb422700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b7bf0 con 0x5555eb3ffcc0 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b5460 con 0x5555eb40ae80 2024-09-12T03:00:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01609bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b32d0 con 0x5555eb4005e0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39556/0 (socket says 172.21.15.46:39556) 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 -- 172.21.15.46:0/3404727600 learned_addr learned my addr 172.21.15.46:0/3404727600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f0156ffd640 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb3fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 -- 172.21.15.46:0/3404727600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 msgr2=0x5555eb3fb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb3fb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 -- 172.21.15.46:0/3404727600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 msgr2=0x5555eb422700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb422700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 -- 172.21.15.46:0/3404727600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eb4063d0 con 0x5555eb3ffcc0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f01577fe640 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c3e50 tx=0x7f014c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8f54b979cabf40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f013d7fa640 1 -- 172.21.15.46:0/3404727600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c052070 con 0x5555eb3ffcc0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f013d7fa640 1 -- 172.21.15.46:0/3404727600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0c8500 con 0x5555eb3ffcc0 2024-09-12T03:00:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.457+0000 7f013d7fa640 1 -- 172.21.15.46:0/3404727600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0ccdc0 con 0x5555eb3ffcc0 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- 172.21.15.46:0/3404727600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 msgr2=0x5555eb408750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c3e50 tx=0x7f014c00c040 comp rx=0 tx=0).stop 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- 172.21.15.46:0/3404727600 shutdown_connections 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb40ae80 0x5555eb422700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb4005e0 0x5555eb3fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 --2- 172.21.15.46:0/3404727600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb3ffcc0 0x5555eb408750 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- 172.21.15.46:0/3404727600 >> 172.21.15.46:0/3404727600 conn(0x5555eb3fc230 msgr2=0x5555eb3faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- 172.21.15.46:0/3404727600 shutdown_connections 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- 172.21.15.46:0/3404727600 wait complete. 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.458+0000 7f01609bb740 1 -- start start 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 Processor -- start 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb3ffcc0 0x5555eb40dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb40ae80 0x5555eb4836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b3920 con 0x5555eb4005e0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b4d90 con 0x5555eb3ffcc0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b3ff0 con 0x5555eb40ae80 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01609bb740 10 client.0 _ll_register_callbacks cb 0x7f01480093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39562/0 (socket says 172.21.15.46:39562) 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 -- 172.21.15.46:0/4069663965 learned_addr learned my addr 172.21.15.46:0/4069663965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0157fff640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb40ae80 0x5555eb4836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f01577fe640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb3ffcc0 0x5555eb40dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 -- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb40ae80 msgr2=0x5555eb4836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb40ae80 0x5555eb4836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 -- 172.21.15.46:0/4069663965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb3ffcc0 msgr2=0x5555eb40dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb3ffcc0 0x5555eb40dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.459+0000 7f0156ffd640 1 -- 172.21.15.46:0/4069663965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eb40e8c0 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0156ffd640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0158072a50 tx=0x7f0148016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b9ec73ffbf161d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480155d0 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014c0c8500 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f0148015770 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f014802e050 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 10 client.16152 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555eb484d60 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0c8500 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f0148025880 con 0x5555eb4005e0 2024-09-12T03:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 10 client.16152 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 20 client.16152 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb487560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 4 client.16152 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.460+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555eb36bd70 con 0x5555eb41aa40 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.461+0000 7f01577fe640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb487560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.461+0000 7f01577fe640 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb487560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4ea8c5631c371b server_cookie=8098037b75bb4200 in_seq=0 out_seq=0 2024-09-12T03:00:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.461+0000 7f0136ffd640 10 client.16152 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.624+0000 7f0136ffd640 1 -- 172.21.15.46:0/4069663965 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555eb36bd70 con 0x5555eb41aa40 2024-09-12T03:00:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.624+0000 7f0136ffd640 10 client.16152 handle_command_reply: tid=0 2024-09-12T03:00:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.624+0000 7f01609bb740 1 client.16152 shutdown 2024-09-12T03:00:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 msgr2=0x5555eb40e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0158072a50 tx=0x7f0148016800 comp rx=0 tx=0).stop 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 shutdown_connections 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5555eb41aa40 0x5555eb487560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f0136ffd640 0 client.16152 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5555eb40ae80 0x5555eb4836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555eb4005e0 0x5555eb40e010 secure :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0158072a50 tx=0x7f0148016800 comp rx=0 tx=0).stop 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 --2- 172.21.15.46:0/4069663965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555eb3ffcc0 0x5555eb40dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.625+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 >> 172.21.15.46:0/4069663965 conn(0x5555eb3fc230 msgr2=0x5555eb406320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.628+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 shutdown_connections 2024-09-12T03:00:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.628+0000 7f01609bb740 1 -- 172.21.15.46:0/4069663965 wait complete. 2024-09-12T03:00:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:00:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:00:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:00:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:00:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.630+0000 7f01609bb740 20 client.16152 trim_cache size 0 max 16384 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 msgr2=0x7f01400a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 0x7f01400a5200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f015000bfd0 tx=0x7f015000d040 comp rx=0 tx=0).stop 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 msgr2=0x7f01581a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f014c004770 tx=0x7f014c00a3b0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 shutdown_connections 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f01400a2d40 0x7f01400a5200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f015819e590 0x7f015819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0158112920 0x7f01581a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 --2- 172.21.15.46:0/2302728123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0158109460 0x7f01581a43e0 secure :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f014c004770 tx=0x7f014c00a3b0 comp rx=0 tx=0).stop 2024-09-12T03:00:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.632+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 >> 172.21.15.46:0/2302728123 conn(0x7f015806d040 msgr2=0x7f0158071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.642+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 shutdown_connections 2024-09-12T03:00:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:33.643+0000 7f015c8e7640 1 -- 172.21.15.46:0/2302728123 wait complete. 2024-09-12T03:00:34.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:34 smithi145 ceph-mon[34674]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-09-12T03:00:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:34 smithi196 ceph-mon[34587]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-09-12T03:00:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:34 smithi046 ceph-mon[30951]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-09-12T03:00:36.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:36 smithi145 ceph-mon[34674]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.10k op/s 2024-09-12T03:00:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:36 smithi196 ceph-mon[34587]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.10k op/s 2024-09-12T03:00:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:36 smithi046 ceph-mon[30951]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.10k op/s 2024-09-12T03:00:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:38 smithi145 ceph-mon[34674]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:00:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:38 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:38.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:38 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:38 smithi196 ceph-mon[34587]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:00:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:38 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:38 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:38.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:38 smithi046 ceph-mon[30951]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:00:38.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:38 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:38.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:38 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:39.611 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:39.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:39.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:00:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:40 smithi196 ceph-mon[34587]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T03:00:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:40 smithi196 ceph-mon[34587]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T03:00:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:40 smithi196 ceph-mon[34587]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-09-12T03:00:40.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:40 smithi046 ceph-mon[30951]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T03:00:40.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:40 smithi046 ceph-mon[30951]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T03:00:40.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:40 smithi046 ceph-mon[30951]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-09-12T03:00:40.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:40 smithi145 ceph-mon[34674]: Adjusting osd_memory_target on smithi046 to 177.0M 2024-09-12T03:00:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:40 smithi145 ceph-mon[34674]: Unable to set osd_memory_target on smithi046 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T03:00:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:40 smithi145 ceph-mon[34674]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-09-12T03:00:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:41 smithi196 ceph-mon[34587]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:00:41.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:41 smithi046 ceph-mon[30951]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:00:41.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:41 smithi145 ceph-mon[34674]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:00:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:43 smithi196 ceph-mon[34587]: pgmap v328: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:44.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:43 smithi046 ceph-mon[30951]: pgmap v328: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:43 smithi145 ceph-mon[34674]: pgmap v328: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2024-09-12T03:00:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:45 smithi196 ceph-mon[34587]: pgmap v329: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-12T03:00:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:45 smithi046 ceph-mon[30951]: pgmap v329: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-12T03:00:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:45 smithi145 ceph-mon[34674]: pgmap v329: 129 pgs: 129 active+clean; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-12T03:00:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.881+0000 7f27f7376640 1 -- 172.21.15.46:0/1050896460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 msgr2=0x7f27f0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.881+0000 7f27f5372640 1 -- 172.21.15.46:0/1050896460 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec035020 con 0x7f27f01091f0 2024-09-12T03:00:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.881+0000 7f27f7376640 1 --2- 172.21.15.46:0/1050896460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f0110250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009900 tx=0x7f27ec02a410 comp rx=0 tx=0).stop 2024-09-12T03:00:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.885+0000 7f27f7376640 1 -- 172.21.15.46:0/1050896460 shutdown_connections 2024-09-12T03:00:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.885+0000 7f27f7376640 1 --2- 172.21.15.46:0/1050896460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.885+0000 7f27f7376640 1 --2- 172.21.15.46:0/1050896460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 0x7f27f0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.885+0000 7f27f7376640 1 --2- 172.21.15.46:0/1050896460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f0110250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.885+0000 7f27f7376640 1 -- 172.21.15.46:0/1050896460 >> 172.21.15.46:0/1050896460 conn(0x7f27f006cbf0 msgr2=0x7f27f006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.887+0000 7f27f7376640 1 -- 172.21.15.46:0/1050896460 shutdown_connections 2024-09-12T03:00:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.887+0000 7f27f7376640 1 -- 172.21.15.46:0/1050896460 wait complete. 2024-09-12T03:00:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 Processor -- start 2024-09-12T03:00:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 -- start start 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f01a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 0x7f27f01a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f01ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0120a20 con 0x7f27f01095d0 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01208a0 con 0x7f27f01091f0 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.888+0000 7f27f7376640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0120ba0 con 0x7f27f011b140 2024-09-12T03:00:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.889+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f01ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.890+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f01ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:39382/0 (socket says 172.21.15.46:39382) 2024-09-12T03:00:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.890+0000 7f27f6b75640 1 -- 172.21.15.46:0/3593696508 learned_addr learned my addr 172.21.15.46:0/3593696508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.890+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f01a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.891+0000 7f27f5b73640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 0x7f27f01a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.893+0000 7f27f6374640 1 -- 172.21.15.46:0/3593696508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 msgr2=0x7f27f01ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.893+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f01ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.893+0000 7f27f6374640 1 -- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 msgr2=0x7f27f01a74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.893+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 0x7f27f01a74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.893+0000 7f27f6374640 1 -- 172.21.15.46:0/3593696508 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f01abfa0 con 0x7f27f01091f0 2024-09-12T03:00:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.895+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f01a6fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c00 tx=0x7f27ec003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.901+0000 7f27e77fe640 1 -- 172.21.15.46:0/3593696508 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec03f070 con 0x7f27f01091f0 2024-09-12T03:00:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.901+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f01ac230 con 0x7f27f01091f0 2024-09-12T03:00:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.901+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01b3b20 con 0x7f27f01091f0 2024-09-12T03:00:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.901+0000 7f27e77fe640 1 -- 172.21.15.46:0/3593696508 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec03bb20 con 0x7f27f01091f0 2024-09-12T03:00:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.901+0000 7f27e77fe640 1 -- 172.21.15.46:0/3593696508 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec035030 con 0x7f27f01091f0 2024-09-12T03:00:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.906+0000 7f27fb449740 1 Processor -- start 2024-09-12T03:00:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 -- start start 2024-09-12T03:00:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92db5e0 0x55fad92e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fad923f2f0 con 0x55fad92db5e0 2024-09-12T03:00:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fad91e7270 con 0x55fad92e5e80 2024-09-12T03:00:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.907+0000 7f27fb449740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fad923b2b0 con 0x55fad92dacc0 2024-09-12T03:00:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:39384/0 (socket says 172.21.15.46:39384) 2024-09-12T03:00:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 -- 172.21.15.46:0/3977569388 learned_addr learned my addr 172.21.15.46:0/3977569388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 -- 172.21.15.46:0/3977569388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 msgr2=0x55fad92e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 -- 172.21.15.46:0/3977569388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92db5e0 msgr2=0x55fad92e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92db5e0 0x55fad92e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27f6b75640 1 -- 172.21.15.46:0/3977569388 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad91e6340 con 0x55fad92dacc0 2024-09-12T03:00:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.909+0000 7f27e77fe640 1 -- 172.21.15.46:0/3593696508 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f27ec042070 con 0x7f27f01091f0 2024-09-12T03:00:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.910+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e8011410 tx=0x7f27e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86bd1351b38e29c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.911+0000 7f27e77fe640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 0x7f27dc0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.911+0000 7f27e77fe640 1 -- 172.21.15.46:0/3593696508 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f27ec0efcd0 con 0x7f27f01091f0 2024-09-12T03:00:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.912+0000 7f27e4bfe640 1 -- 172.21.15.46:0/3977569388 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8012770 con 0x55fad92dacc0 2024-09-12T03:00:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.912+0000 7f27e4bfe640 1 -- 172.21.15.46:0/3977569388 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e8012dc0 con 0x55fad92dacc0 2024-09-12T03:00:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.913+0000 7f27fb449740 1 -- 172.21.15.46:0/3977569388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 msgr2=0x55fad92db0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.913+0000 7f27fb449740 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e8011410 tx=0x7f27e80118d0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.917+0000 7f27fb449740 1 -- 172.21.15.46:0/3977569388 shutdown_connections 2024-09-12T03:00:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.917+0000 7f27fb449740 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.917+0000 7f27fb449740 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92db5e0 0x55fad92e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.917+0000 7f27fb449740 1 --2- 172.21.15.46:0/3977569388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92db0a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.917+0000 7f27fb449740 1 -- 172.21.15.46:0/3977569388 >> 172.21.15.46:0/3977569388 conn(0x55fad92d7230 msgr2=0x55fad92d7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.919+0000 7f27fb449740 1 -- 172.21.15.46:0/3977569388 shutdown_connections 2024-09-12T03:00:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.920+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 0x7f27dc0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.923+0000 7f27fb449740 1 -- 172.21.15.46:0/3977569388 wait complete. 2024-09-12T03:00:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.924+0000 7f27fb449740 1 -- start start 2024-09-12T03:00:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.924+0000 7f27fb449740 1 Processor -- start 2024-09-12T03:00:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.928+0000 7f27fb449740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92dacc0 0x55fad92e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.928+0000 7f27fb449740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.928+0000 7f27fb449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.928+0000 7f27fb449740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fad92e8aa0 con 0x55fad92dacc0 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.928+0000 7f27fb449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fad91c8290 con 0x55fad92e5e80 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27fb449740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fad92e8c20 con 0x55fad92db5e0 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:39388/0 (socket says 172.21.15.46:39388) 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 -- 172.21.15.46:0/3615511805 learned_addr learned my addr 172.21.15.46:0/3615511805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 -- 172.21.15.46:0/3615511805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 msgr2=0x55fad92e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 -- 172.21.15.46:0/3615511805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92dacc0 msgr2=0x55fad92e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92dacc0 0x55fad92e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 -- 172.21.15.46:0/3615511805 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad8f52210 con 0x55fad92db5e0 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.929+0000 7f27f5b73640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27e000e450 tx=0x7f27e000e910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a839dd893d19de59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.930+0000 7f27fb449740 10 client.0 _ll_register_callbacks cb 0x55fad91f9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.930+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e8012dc0 con 0x55fad92db5e0 2024-09-12T03:00:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.932+0000 7f27f6374640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 0x7f27dc0a53c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fad92d8090 tx=0x7f27ec005b70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.932+0000 7f27d9ffb640 10 client.25693 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:00:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.932+0000 7f27d9ffb640 1 -- 172.21.15.46:0/3615511805 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0017070 con 0x55fad92db5e0 2024-09-12T03:00:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.933+0000 7f27d9ffb640 1 -- 172.21.15.46:0/3615511805 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f27e0013680 con 0x55fad92db5e0 2024-09-12T03:00:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.933+0000 7f27d9ffb640 1 -- 172.21.15.46:0/3615511805 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27e0023050 con 0x55fad92db5e0 2024-09-12T03:00:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.934+0000 7f27fb449740 10 client.25693 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.934+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55fad901e620 con 0x55fad92db5e0 2024-09-12T03:00:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27d9ffb640 1 -- 172.21.15.46:0/3615511805 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f27e0013880 con 0x55fad92db5e0 2024-09-12T03:00:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 10 client.25693 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 20 client.25693 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad92f61f0 0x55fad92f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 4 client.25693 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad924ca20 con 0x55fad92f61f0 2024-09-12T03:00:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.935+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad92f61f0 0x55fad92f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.936+0000 7f27f6b75640 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad92f61f0 0x55fad92f86b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77c09fce2fa131f server_cookie=b3cbfdcfdbdb93be in_seq=0 out_seq=0 2024-09-12T03:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.943+0000 7f27d9ffb640 10 client.25693 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.943+0000 7f27d9ffb640 1 -- 172.21.15.46:0/3615511805 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fad924ca20 con 0x55fad92f61f0 2024-09-12T03:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.943+0000 7f27d9ffb640 10 client.25693 handle_command_reply: tid=0 2024-09-12T03:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.943+0000 7f27fb449740 1 client.25693 shutdown 2024-09-12T03:00:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.945+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 msgr2=0x55fad92e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.945+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27e000e450 tx=0x7f27e000e910 comp rx=0 tx=0).stop 2024-09-12T03:00:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 shutdown_connections 2024-09-12T03:00:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad92f61f0 0x55fad92f86b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e5e80 0x55fad92e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92db5e0 0x55fad92e3200 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27e000e450 tx=0x7f27e000e910 comp rx=0 tx=0).stop 2024-09-12T03:00:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 --2- 172.21.15.46:0/3615511805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92dacc0 0x55fad92e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.946+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 >> 172.21.15.46:0/3615511805 conn(0x55fad92d7230 msgr2=0x55fad92dcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.948+0000 7f27d9ffb640 0 client.25693 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.953+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 shutdown_connections 2024-09-12T03:00:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.956+0000 7f27fb449740 1 -- 172.21.15.46:0/3615511805 wait complete. 2024-09-12T03:00:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:46.958+0000 7f27fb449740 20 client.25693 trim_cache size 0 max 16384 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.034+0000 7f27fb449740 1 Processor -- start 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 -- start start 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92e5e80 0x55fad92fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e19d0 0x55fad92fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40128b0 con 0x55fad92e19d0 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27d4010120 con 0x55fad92dacc0 2024-09-12T03:00:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.035+0000 7f27fb449740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27d400df90 con 0x55fad92e5e80 2024-09-12T03:00:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.037+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.037+0000 7f27f6b75640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:46718/0 (socket says 172.21.15.46:46718) 2024-09-12T03:00:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.037+0000 7f27f6b75640 1 -- 172.21.15.46:0/488092952 learned_addr learned my addr 172.21.15.46:0/488092952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.037+0000 7f27f6b75640 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e19d0 0x55fad92fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.037+0000 7f27f5b73640 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92e5e80 0x55fad92fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.038+0000 7f27f6b75640 1 -- 172.21.15.46:0/488092952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92e5e80 msgr2=0x55fad92fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.038+0000 7f27f6b75640 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92e5e80 0x55fad92fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.038+0000 7f27f6b75640 1 -- 172.21.15.46:0/488092952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e19d0 msgr2=0x55fad92fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.038+0000 7f27f6b75640 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e19d0 0x55fad92fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.038+0000 7f27f6b75640 1 -- 172.21.15.46:0/488092952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad92f2bd0 con 0x55fad92dacc0 2024-09-12T03:00:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.039+0000 7f27f6b75640 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e800fce0 tx=0x7f27e8012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8403fdb033fab7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.039+0000 7f27dbfff640 1 -- 172.21.15.46:0/488092952 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e800f5e0 con 0x55fad92dacc0 2024-09-12T03:00:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.039+0000 7f27dbfff640 1 -- 172.21.15.46:0/488092952 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e8012570 con 0x55fad92dacc0 2024-09-12T03:00:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 -- 172.21.15.46:0/488092952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 msgr2=0x55fad92e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27e800fce0 tx=0x7f27e8012f90 comp rx=0 tx=0).stop 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 -- 172.21.15.46:0/488092952 shutdown_connections 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e19d0 0x55fad92fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92e5e80 0x55fad92fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.040+0000 7f27fb449740 1 --2- 172.21.15.46:0/488092952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92dacc0 0x55fad92e3570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.041+0000 7f27fb449740 1 -- 172.21.15.46:0/488092952 >> 172.21.15.46:0/488092952 conn(0x55fad92d7230 msgr2=0x55fad92e4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.041+0000 7f27fb449740 1 -- 172.21.15.46:0/488092952 shutdown_connections 2024-09-12T03:00:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.041+0000 7f27fb449740 1 -- 172.21.15.46:0/488092952 wait complete. 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.041+0000 7f27fb449740 1 -- start start 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.041+0000 7f27fb449740 1 Processor -- start 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 0x55fad92e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 0x55fad92e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27d400e5e0 con 0x55fad92e5e80 2024-09-12T03:00:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f27d400fa50 con 0x55fad92e19d0 2024-09-12T03:00:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27d400ecb0 con 0x55fad92dacc0 2024-09-12T03:00:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.042+0000 7f27fb449740 10 client.0 _ll_register_callbacks cb 0x7f27e0009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.044+0000 7f27f5b73640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 0x55fad92e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.044+0000 7f27f5b73640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 0x55fad92e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:46728/0 (socket says 172.21.15.46:46728) 2024-09-12T03:00:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.044+0000 7f27f5b73640 1 -- 172.21.15.46:0/4238175651 learned_addr learned my addr 172.21.15.46:0/4238175651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:00:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 0x55fad92e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 -- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 msgr2=0x55fad92ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 -- 172.21.15.46:0/4238175651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 msgr2=0x55fad92e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 0x55fad92e6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.045+0000 7f27f6b75640 1 -- 172.21.15.46:0/4238175651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad92e7590 con 0x55fad92e5e80 2024-09-12T03:00:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.046+0000 7f27f6b75640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 0x55fad92e6fd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f27e8010010 tx=0x7f27e8009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa460088aa861575 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.046+0000 7f27d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:00:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.047+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8004d90 con 0x55fad92e5e80 2024-09-12T03:00:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.047+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e8012570 con 0x55fad92e5e80 2024-09-12T03:00:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.047+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7f27e801a990 con 0x55fad92e5e80 2024-09-12T03:00:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.049+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27e801ab90 con 0x55fad92e5e80 2024-09-12T03:00:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.049+0000 7f27fb449740 10 client.16161 fetch_fsmap learned FSMap version 33 2024-09-12T03:00:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.049+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fad92ecdd0 con 0x55fad92e5e80 2024-09-12T03:00:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.050+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e8012570 con 0x55fad92e5e80 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f27e8005370 con 0x55fad92e5e80 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 10 client.16161 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 20 client.16161 populate_metadata read hostname 'smithi046' 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad9372370 0x55fad9374790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 4 client.16161 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:00:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.051+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fad9246d70 con 0x55fad9372370 2024-09-12T03:00:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.054+0000 7f27f5b73640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad9372370 0x55fad9374790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.055+0000 7f27f5b73640 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad9372370 0x55fad9374790 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ef5bb41f20f780 server_cookie=30db97827e0248a3 in_seq=0 out_seq=0 2024-09-12T03:00:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.056+0000 7f27d9ffb640 10 client.16161 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.075+0000 7f27d9ffb640 1 -- 172.21.15.46:0/4238175651 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fad9246d70 con 0x55fad9372370 2024-09-12T03:00:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.075+0000 7f27d9ffb640 10 client.16161 handle_command_reply: tid=0 2024-09-12T03:00:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.075+0000 7f27fb449740 1 client.16161 shutdown 2024-09-12T03:00:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 msgr2=0x55fad92e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 0x55fad92e6fd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f27e8010010 tx=0x7f27e8009610 comp rx=0 tx=0).stop 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 shutdown_connections 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55fad9372370 0x55fad9374790 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fad92e5e80 0x55fad92e6fd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fad92e19d0 0x55fad92e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 --2- 172.21.15.46:0/4238175651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fad92dacc0 0x55fad92ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 >> 172.21.15.46:0/4238175651 conn(0x55fad92d7230 msgr2=0x55fad92d6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27d9ffb640 0 client.16161 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:00:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.082+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 shutdown_connections 2024-09-12T03:00:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.083+0000 7f27fb449740 1 -- 172.21.15.46:0/4238175651 wait complete. 2024-09-12T03:00:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.083+0000 7f27fb449740 20 client.16161 trim_cache size 0 max 16384 2024-09-12T03:00:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:00:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:00:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:00:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:00:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.095+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 msgr2=0x7f27dc0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.095+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 0x7f27dc0a53c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fad92d8090 tx=0x7f27ec005b70 comp rx=0 tx=0).stop 2024-09-12T03:00:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.095+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 msgr2=0x7f27f01a6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.095+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f01a6fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c00 tx=0x7f27ec003c10 comp rx=0 tx=0).stop 2024-09-12T03:00:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.096+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 shutdown_connections 2024-09-12T03:00:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.096+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f27dc0a2f00 0x7f27dc0a53c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.096+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f011b140 0x7f27f01ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.097+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f01095d0 0x7f27f01a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.097+0000 7f27f7376640 1 --2- 172.21.15.46:0/3593696508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27f01091f0 0x7f27f01a6fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.097+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 >> 172.21.15.46:0/3593696508 conn(0x7f27f006cbf0 msgr2=0x7f27f011b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.103+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 shutdown_connections 2024-09-12T03:00:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:00:47.103+0000 7f27f7376640 1 -- 172.21.15.46:0/3593696508 wait complete. 2024-09-12T03:00:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:47 smithi196 ceph-mon[34587]: pgmap v330: 129 pgs: 129 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:00:48.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:47 smithi046 ceph-mon[30951]: pgmap v330: 129 pgs: 129 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:00:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:47 smithi145 ceph-mon[34674]: pgmap v330: 129 pgs: 129 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:00:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:49 smithi196 ceph-mon[34587]: pgmap v331: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-12T03:00:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:49 smithi046 ceph-mon[30951]: pgmap v331: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-12T03:00:50.113 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:49 smithi145 ceph-mon[34674]: pgmap v331: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-09-12T03:00:51.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:51 smithi196 ceph-mon[34587]: pgmap v332: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:00:52.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:51 smithi046 ceph-mon[30951]: pgmap v332: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:00:52.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:51 smithi145 ceph-mon[34674]: pgmap v332: 129 pgs: 129 active+clean; 14 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:00:52.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:52 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:00:53.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:52 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:00:53.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:52 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:00:53.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:53 smithi196 ceph-mon[34587]: pgmap v333: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 5.24k op/s 2024-09-12T03:00:54.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:53 smithi046 ceph-mon[30951]: pgmap v333: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 5.24k op/s 2024-09-12T03:00:54.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:53 smithi145 ceph-mon[34674]: pgmap v333: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 21 MiB/s wr, 5.24k op/s 2024-09-12T03:00:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:55 smithi046 ceph-mon[30951]: pgmap v334: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:56.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:55 smithi145 ceph-mon[34674]: pgmap v334: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:56.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:55 smithi196 ceph-mon[34587]: pgmap v334: 129 pgs: 129 active+clean; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:00:58.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:57 smithi196 ceph-mon[34587]: pgmap v335: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-09-12T03:00:58.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:57 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:00:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:57 smithi046 ceph-mon[30951]: pgmap v335: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-09-12T03:00:58.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:57 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:00:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:57 smithi145 ceph-mon[34674]: pgmap v335: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-09-12T03:00:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:57 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:00:59 smithi046 ceph-mon[30951]: pgmap v336: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2024-09-12T03:01:00.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:00:59 smithi145 ceph-mon[34674]: pgmap v336: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2024-09-12T03:01:00.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:00:59 smithi196 ceph-mon[34587]: pgmap v336: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2024-09-12T03:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.330+0000 7ffb504e6640 1 -- 172.21.15.46:0/3340908481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb4811b140 msgr2=0x7ffb4811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.330+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3340908481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb4811b140 0x7ffb4811d530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009930 tx=0x7ffb4402a480 comp rx=0 tx=0).stop 2024-09-12T03:01:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.332+0000 7ffb504e6640 1 -- 172.21.15.46:0/3340908481 shutdown_connections 2024-09-12T03:01:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.332+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3340908481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb4811b140 0x7ffb4811d530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.332+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3340908481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb481095d0 0x7ffb48109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.332+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3340908481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481091f0 0x7ffb48110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.332+0000 7ffb504e6640 1 -- 172.21.15.46:0/3340908481 >> 172.21.15.46:0/3340908481 conn(0x7ffb4806ce00 msgr2=0x7ffb4806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.334+0000 7ffb504e6640 1 -- 172.21.15.46:0/3340908481 shutdown_connections 2024-09-12T03:01:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.337+0000 7ffb504e6640 1 -- 172.21.15.46:0/3340908481 wait complete. 2024-09-12T03:01:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.340+0000 7ffb504e6640 1 Processor -- start 2024-09-12T03:01:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.340+0000 7ffb504e6640 1 -- start start 2024-09-12T03:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.340+0000 7ffb504e6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.340+0000 7ffb504e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481095d0 0x7ffb481a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.340+0000 7ffb504e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4811b140 0x7ffb481ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb504e6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48120ba0 con 0x7ffb481095d0 2024-09-12T03:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb504e6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48120a20 con 0x7ffb481091f0 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb504e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48120d20 con 0x7ffb4811b140 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:45468/0 (socket says 172.21.15.46:45468) 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 -- 172.21.15.46:0/3554450281 learned_addr learned my addr 172.21.15.46:0/3554450281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4da81640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481095d0 0x7ffb481a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4811b140 0x7ffb481ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 -- 172.21.15.46:0/3554450281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4811b140 msgr2=0x7ffb481ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4811b140 0x7ffb481ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 -- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481095d0 msgr2=0x7ffb481a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481095d0 0x7ffb481a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.341+0000 7ffb4e282640 1 -- 172.21.15.46:0/3554450281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb481abf80 con 0x7ffb481091f0 2024-09-12T03:01:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.343+0000 7ffb4e282640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800ea60 tx=0x7ffb3800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.344+0000 7ffb3f7fe640 1 -- 172.21.15.46:0/3554450281 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800cde0 con 0x7ffb481091f0 2024-09-12T03:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.345+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb481ac270 con 0x7ffb481091f0 2024-09-12T03:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.345+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb481b3b00 con 0x7ffb481091f0 2024-09-12T03:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.345+0000 7ffb3f7fe640 1 -- 172.21.15.46:0/3554450281 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb380035f0 con 0x7ffb481091f0 2024-09-12T03:01:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.345+0000 7ffb3f7fe640 1 -- 172.21.15.46:0/3554450281 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38010690 con 0x7ffb481091f0 2024-09-12T03:01:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.349+0000 7ffb5327d740 1 Processor -- start 2024-09-12T03:01:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 -- start start 2024-09-12T03:01:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2de1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de2410 0x55ddf2dec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2d462f0 con 0x55ddf2de2410 2024-09-12T03:01:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2cee270 con 0x55ddf2deccb0 2024-09-12T03:01:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.353+0000 7ffb5327d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2d422b0 con 0x55ddf2de1af0 2024-09-12T03:01:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.354+0000 7ffb3f7fe640 1 -- 172.21.15.46:0/3554450281 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb38004020 con 0x7ffb481091f0 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.355+0000 7ffb3f7fe640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 0x7ffb340a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.356+0000 7ffb3f7fe640 1 -- 172.21.15.46:0/3554450281 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7ffb380cfa70 con 0x7ffb481091f0 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.356+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.356+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:45472/0 (socket says 172.21.15.46:45472) 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.356+0000 7ffb4da81640 1 -- 172.21.15.46:0/2087017345 learned_addr learned my addr 172.21.15.46:0/2087017345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.357+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2de1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.359+0000 7ffb4da81640 1 -- 172.21.15.46:0/2087017345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 msgr2=0x55ddf2de1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.359+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2de1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.360+0000 7ffb4da81640 1 -- 172.21.15.46:0/2087017345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de2410 msgr2=0x55ddf2dec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.360+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de2410 0x55ddf2dec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.360+0000 7ffb4da81640 1 -- 172.21.15.46:0/2087017345 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf2ced340 con 0x55ddf2deccb0 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.361+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2de1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.361+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 0x7ffb340a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.362+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 0x7ffb340a5480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006010 tx=0x7ffb44009260 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.363+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009fd0 tx=0x7ffb40010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896efd2e85a65375 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.370+0000 7ffb3cbfe640 1 -- 172.21.15.46:0/2087017345 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40003b70 con 0x55ddf2deccb0 2024-09-12T03:01:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.370+0000 7ffb3cbfe640 1 -- 172.21.15.46:0/2087017345 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40015070 con 0x55ddf2deccb0 2024-09-12T03:01:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 -- 172.21.15.46:0/2087017345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 msgr2=0x55ddf2def0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009fd0 tx=0x7ffb40010860 comp rx=0 tx=0).stop 2024-09-12T03:01:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 -- 172.21.15.46:0/2087017345 shutdown_connections 2024-09-12T03:01:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2def0a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de2410 0x55ddf2dec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2087017345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2de1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.371+0000 7ffb5327d740 1 -- 172.21.15.46:0/2087017345 >> 172.21.15.46:0/2087017345 conn(0x55ddf2dde060 msgr2=0x55ddf2dde440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.372+0000 7ffb5327d740 1 -- 172.21.15.46:0/2087017345 shutdown_connections 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.372+0000 7ffb5327d740 1 -- 172.21.15.46:0/2087017345 wait complete. 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.372+0000 7ffb5327d740 1 -- start start 2024-09-12T03:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.372+0000 7ffb5327d740 1 Processor -- start 2024-09-12T03:01:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2de1af0 0x55ddf2de9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2deccb0 0x55ddf2dea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2e008e0 con 0x55ddf2deccb0 2024-09-12T03:01:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2ccf290 con 0x55ddf2de1af0 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddf2e00a60 con 0x55ddf2de2410 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb5327d740 10 client.0 _ll_register_callbacks cb 0x55ddf2d00140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:53906/0 (socket says 172.21.15.46:53906) 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.373+0000 7ffb4da81640 1 -- 172.21.15.46:0/276020063 learned_addr learned my addr 172.21.15.46:0/276020063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 -- 172.21.15.46:0/276020063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2de1af0 msgr2=0x55ddf2de9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2de1af0 0x55ddf2de9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 -- 172.21.15.46:0/276020063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2deccb0 msgr2=0x55ddf2dea600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2deccb0 0x55ddf2dea600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 -- 172.21.15.46:0/276020063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf2a59210 con 0x55ddf2de2410 2024-09-12T03:01:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb4da81640 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006010 tx=0x7ffb40011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7960687b6b8f95a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.374+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4002a060 con 0x55ddf2de2410 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.375+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb40015070 con 0x55ddf2de2410 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.375+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7ffb40003b80 con 0x55ddf2de2410 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.375+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffb40031050 con 0x55ddf2de2410 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.376+0000 7ffb5327d740 10 client.25708 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.376+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddf2b25620 con 0x55ddf2de2410 2024-09-12T03:01:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40015070 con 0x55ddf2de2410 2024-09-12T03:01:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ffb4001a590 con 0x55ddf2de2410 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 10 client.25708 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 20 client.25708 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e0e120 0x55ddf2e105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 4 client.25708 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.379+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf2d53a20 con 0x55ddf2e0e120 2024-09-12T03:01:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.384+0000 7ffb4e282640 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e0e120 0x55ddf2e105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.389+0000 7ffb4e282640 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e0e120 0x55ddf2e105a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59aa3a840728df6 server_cookie=b7db16014199eab7 in_seq=0 out_seq=0 2024-09-12T03:01:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.390+0000 7ffb31ffb640 10 client.25708 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.391+0000 7ffb31ffb640 1 -- 172.21.15.46:0/276020063 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddf2d53a20 con 0x55ddf2e0e120 2024-09-12T03:01:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.391+0000 7ffb31ffb640 10 client.25708 handle_command_reply: tid=0 2024-09-12T03:01:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.391+0000 7ffb5327d740 1 client.25708 shutdown 2024-09-12T03:01:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.396+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 msgr2=0x55ddf2dea0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.396+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006010 tx=0x7ffb40011d50 comp rx=0 tx=0).stop 2024-09-12T03:01:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 shutdown_connections 2024-09-12T03:01:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e0e120 0x55ddf2e105a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2deccb0 0x55ddf2dea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de2410 0x55ddf2dea0c0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006010 tx=0x7ffb40011d50 comp rx=0 tx=0).stop 2024-09-12T03:01:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 --2- 172.21.15.46:0/276020063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2de1af0 0x55ddf2de9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 >> 172.21.15.46:0/276020063 conn(0x55ddf2dde060 msgr2=0x55ddf2de3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.398+0000 7ffb31ffb640 0 client.25708 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.402+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 shutdown_connections 2024-09-12T03:01:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.407+0000 7ffb5327d740 1 -- 172.21.15.46:0/276020063 wait complete. 2024-09-12T03:01:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.408+0000 7ffb5327d740 20 client.25708 trim_cache size 0 max 16384 2024-09-12T03:01:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.447+0000 7ffb5327d740 1 Processor -- start 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 -- start start 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de1af0 0x55ddf2debaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de8c50 0x55ddf2e163b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0128c0 con 0x55ddf2de1af0 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c010100 con 0x55ddf2deccb0 2024-09-12T03:01:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.449+0000 7ffb5327d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c00df70 con 0x55ddf2de8c50 2024-09-12T03:01:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.450+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.450+0000 7ffb4ea83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de8c50 0x55ddf2e163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.450+0000 7ffb4da81640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:45484/0 (socket says 172.21.15.46:45484) 2024-09-12T03:01:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.450+0000 7ffb4da81640 1 -- 172.21.15.46:0/2269748184 learned_addr learned my addr 172.21.15.46:0/2269748184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.450+0000 7ffb4e282640 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de1af0 0x55ddf2debaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.451+0000 7ffb4da81640 1 -- 172.21.15.46:0/2269748184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de8c50 msgr2=0x55ddf2e163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.451+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de8c50 0x55ddf2e163b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.451+0000 7ffb4da81640 1 -- 172.21.15.46:0/2269748184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de1af0 msgr2=0x55ddf2debaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.451+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de1af0 0x55ddf2debaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.451+0000 7ffb4da81640 1 -- 172.21.15.46:0/2269748184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf2de1920 con 0x55ddf2deccb0 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.452+0000 7ffb4da81640 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb400097a0 tx=0x7ffb40011a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cc68735c8a0558 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.452+0000 7ffb33fff640 1 -- 172.21.15.46:0/2269748184 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4002a030 con 0x55ddf2deccb0 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.452+0000 7ffb33fff640 1 -- 172.21.15.46:0/2269748184 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40015070 con 0x55ddf2deccb0 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.452+0000 7ffb33fff640 1 -- 172.21.15.46:0/2269748184 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40005340 con 0x55ddf2deccb0 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.453+0000 7ffb5327d740 1 -- 172.21.15.46:0/2269748184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 msgr2=0x55ddf2e15e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.453+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb400097a0 tx=0x7ffb40011a60 comp rx=0 tx=0).stop 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.456+0000 7ffb5327d740 1 -- 172.21.15.46:0/2269748184 shutdown_connections 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.456+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de8c50 0x55ddf2e163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.456+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2e15e70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.456+0000 7ffb5327d740 1 --2- 172.21.15.46:0/2269748184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de1af0 0x55ddf2debaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.456+0000 7ffb5327d740 1 -- 172.21.15.46:0/2269748184 >> 172.21.15.46:0/2269748184 conn(0x55ddf2dde060 msgr2=0x55ddf2deb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.457+0000 7ffb5327d740 1 -- 172.21.15.46:0/2269748184 shutdown_connections 2024-09-12T03:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.459+0000 7ffb5327d740 1 -- 172.21.15.46:0/2269748184 wait complete. 2024-09-12T03:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.459+0000 7ffb5327d740 1 -- start start 2024-09-12T03:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.459+0000 7ffb5327d740 1 Processor -- start 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2e05fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de8c50 0x55ddf2e06500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2de61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c00e5c0 con 0x55ddf2de8c50 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c00fa30 con 0x55ddf2deccb0 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c00ec90 con 0x55ddf2de1af0 2024-09-12T03:01:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.463+0000 7ffb5327d740 10 client.0 _ll_register_callbacks cb 0x7ffb40009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.465+0000 7ffb4e282640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2e05fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.465+0000 7ffb4e282640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2e05fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:53934/0 (socket says 172.21.15.46:53934) 2024-09-12T03:01:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.465+0000 7ffb4e282640 1 -- 172.21.15.46:0/4160833630 learned_addr learned my addr 172.21.15.46:0/4160833630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.468+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2de61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.468+0000 7ffb4da81640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de8c50 0x55ddf2e06500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.468+0000 7ffb4ea83640 1 -- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 msgr2=0x55ddf2e05fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.468+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2e05fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.468+0000 7ffb4ea83640 1 -- 172.21.15.46:0/4160833630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de8c50 msgr2=0x55ddf2e06500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.469+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de8c50 0x55ddf2e06500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.469+0000 7ffb4ea83640 1 -- 172.21.15.46:0/4160833630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf2de6840 con 0x55ddf2deccb0 2024-09-12T03:01:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.471+0000 7ffb4ea83640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2de61a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009850 tx=0x7ffb4402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46928dad6515b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.473+0000 7ffb31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:01:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.473+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4404b840 con 0x55ddf2deccb0 2024-09-12T03:01:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.473+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9423+0+0 (secure 0 0 0) 0x7ffb4404b9e0 con 0x55ddf2deccb0 2024-09-12T03:01:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.473+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb40015070 con 0x55ddf2deccb0 2024-09-12T03:01:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.474+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffb4404bbe0 con 0x55ddf2deccb0 2024-09-12T03:01:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.475+0000 7ffb5327d740 10 client.25748 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.475+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddf2e78770 con 0x55ddf2deccb0 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.477+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40015070 con 0x55ddf2deccb0 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.477+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ffb4403d420 con 0x55ddf2deccb0 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 10 client.25748 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 20 client.25748 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e78e80 0x55ddf2e7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 4 client.25748 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:01:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.478+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddf2d4dd70 con 0x55ddf2e78e80 2024-09-12T03:01:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.480+0000 7ffb4e282640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e78e80 0x55ddf2e7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.484+0000 7ffb4e282640 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e78e80 0x55ddf2e7b2a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b7bab697aab580 server_cookie=645c00ab16f1619e in_seq=0 out_seq=0 2024-09-12T03:01:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.484+0000 7ffb31ffb640 10 client.25748 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.895+0000 7ffb31ffb640 1 -- 172.21.15.46:0/4160833630 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddf2d4dd70 con 0x55ddf2e78e80 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.895+0000 7ffb31ffb640 10 client.25748 handle_command_reply: tid=0 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 client.25748 shutdown 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 msgr2=0x55ddf2de61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2de61a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009850 tx=0x7ffb4402a480 comp rx=0 tx=0).stop 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 shutdown_connections 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55ddf2e78e80 0x55ddf2e7b2a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ddf2deccb0 0x55ddf2de61a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddf2de8c50 0x55ddf2e06500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 --2- 172.21.15.46:0/4160833630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf2de1af0 0x55ddf2e05fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.896+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 >> 172.21.15.46:0/4160833630 conn(0x55ddf2dde060 msgr2=0x55ddf2ddd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.897+0000 7ffb31ffb640 0 client.25748 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.897+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 shutdown_connections 2024-09-12T03:01:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.897+0000 7ffb5327d740 1 -- 172.21.15.46:0/4160833630 wait complete. 2024-09-12T03:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.901+0000 7ffb5327d740 20 client.25748 trim_cache size 0 max 16384 2024-09-12T03:01:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:01:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:01:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:01:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:01:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.910+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 msgr2=0x7ffb340a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.910+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 0x7ffb340a5480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006010 tx=0x7ffb44009260 comp rx=0 tx=0).stop 2024-09-12T03:01:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.910+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 msgr2=0x7ffb481a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.910+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800ea60 tx=0x7ffb3800ef20 comp rx=0 tx=0).stop 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.914+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 shutdown_connections 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.915+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ffb340a2fc0 0x7ffb340a5480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.915+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4811b140 0x7ffb481ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.915+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb481095d0 0x7ffb481a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.915+0000 7ffb504e6640 1 --2- 172.21.15.46:0/3554450281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb481091f0 0x7ffb481a6f90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.915+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 >> 172.21.15.46:0/3554450281 conn(0x7ffb4806ce00 msgr2=0x7ffb4811b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.916+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 shutdown_connections 2024-09-12T03:01:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:00.916+0000 7ffb504e6640 1 -- 172.21.15.46:0/3554450281 wait complete. 2024-09-12T03:01:02.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:01 smithi046 ceph-mon[30951]: pgmap v337: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:01:02.040 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:01 smithi196 ceph-mon[34587]: pgmap v337: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:01:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:01 smithi145 ceph-mon[34674]: pgmap v337: 129 pgs: 129 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:01:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:03 smithi145 ceph-mon[34674]: pgmap v338: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-09-12T03:01:04.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:03 smithi145 ceph-mon[34674]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:01:04.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:03 smithi196 ceph-mon[34587]: pgmap v338: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-09-12T03:01:04.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:03 smithi196 ceph-mon[34587]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:01:04.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:03 smithi046 ceph-mon[30951]: pgmap v338: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-09-12T03:01:04.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:03 smithi046 ceph-mon[30951]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:01:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:05 smithi145 ceph-mon[34674]: pgmap v340: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:01:06.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:05 smithi196 ceph-mon[34587]: pgmap v340: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:01:06.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:05 smithi046 ceph-mon[30951]: pgmap v340: 129 pgs: 129 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:01:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:07 smithi145 ceph-mon[34674]: pgmap v341: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:07 smithi196 ceph-mon[34587]: pgmap v341: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:08.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:07 smithi046 ceph-mon[30951]: pgmap v341: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:10.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:09 smithi145 ceph-mon[34674]: pgmap v342: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:10.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:09 smithi196 ceph-mon[34587]: pgmap v342: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:10.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:09 smithi046 ceph-mon[30951]: pgmap v342: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:12.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:11 smithi145 ceph-mon[34674]: pgmap v343: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:12.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:11 smithi196 ceph-mon[34587]: pgmap v343: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:12.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:11 smithi046 ceph-mon[30951]: pgmap v343: 129 pgs: 17 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 74 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:14.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:13 smithi145 ceph-mon[34674]: pgmap v344: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:01:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.203+0000 7f67dd98d640 1 -- 172.21.15.46:0/1968460953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 msgr2=0x7f67d811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.203+0000 7f67dd98d640 1 --2- 172.21.15.46:0/1968460953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d811d530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009930 tx=0x7f67cc02a420 comp rx=0 tx=0).stop 2024-09-12T03:01:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 -- 172.21.15.46:0/1968460953 shutdown_connections 2024-09-12T03:01:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 --2- 172.21.15.46:0/1968460953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d811d530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 --2- 172.21.15.46:0/1968460953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67d81095d0 0x7f67d8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 --2- 172.21.15.46:0/1968460953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 -- 172.21.15.46:0/1968460953 >> 172.21.15.46:0/1968460953 conn(0x7f67d806ce00 msgr2=0x7f67d806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.206+0000 7f67dd98d640 1 -- 172.21.15.46:0/1968460953 shutdown_connections 2024-09-12T03:01:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.207+0000 7f67dd98d640 1 -- 172.21.15.46:0/1968460953 wait complete. 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.207+0000 7f67dd98d640 1 Processor -- start 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 -- start start 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d811a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67d81095d0 0x7f67d811abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d8114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81111e0 con 0x7f67d81091f0 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8111060 con 0x7f67d81095d0 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67dd98d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8111360 con 0x7f67d811b140 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.208+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d811a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.209+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d811a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35688/0 (socket says 172.21.15.46:35688) 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.209+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2332758284 learned_addr learned my addr 172.21.15.46:0/2332758284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.209+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d8114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 -- 172.21.15.46:0/2332758284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67d81095d0 msgr2=0x7f67d811abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67d81095d0 0x7f67d811abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 -- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 msgr2=0x7f67d811a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d811a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 -- 172.21.15.46:0/2332758284 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81151e0 con 0x7f67d811b140 2024-09-12T03:01:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.210+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d8114920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009850 tx=0x7f67cc003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.211+0000 7f67dc98b640 1 -- 172.21.15.46:0/2332758284 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc03f070 con 0x7f67d811b140 2024-09-12T03:01:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.211+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d8115470 con 0x7f67d811b140 2024-09-12T03:01:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.211+0000 7f67dc98b640 1 -- 172.21.15.46:0/2332758284 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67cc031b50 con 0x7f67d811b140 2024-09-12T03:01:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.211+0000 7f67dc98b640 1 -- 172.21.15.46:0/2332758284 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc035030 con 0x7f67d811b140 2024-09-12T03:01:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.211+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d807a630 con 0x7f67d811b140 2024-09-12T03:01:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.215+0000 7f67e0800740 1 Processor -- start 2024-09-12T03:01:14.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:13 smithi196 ceph-mon[34587]: pgmap v344: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:01:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67dc98b640 1 -- 172.21.15.46:0/2332758284 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f67cc042070 con 0x7f67d811b140 2024-09-12T03:01:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 -- start start 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1faaf0 0x55aaca1faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca205cb0 0x55aaca2080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aaca15f2f0 con 0x55aaca1faaf0 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aaca107270 con 0x55aaca1fb410 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.225+0000 7f67e0800740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aaca15b2b0 con 0x55aaca205cb0 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.226+0000 7f67dc98b640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 0x7f67c00a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.226+0000 7f67dc98b640 1 -- 172.21.15.46:0/2332758284 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9555+0+0 (secure 0 0 0) 0x7f67cc0efe70 con 0x7f67d811b140 2024-09-12T03:01:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.228+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.232+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 0x7f67c00a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.232+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50776/0 (socket says 172.21.15.46:50776) 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.232+0000 7f67d6ffd640 1 -- 172.21.15.46:0/3149369737 learned_addr learned my addr 172.21.15.46:0/3149369737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.233+0000 7f67d6ffd640 1 -- 172.21.15.46:0/3149369737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca205cb0 msgr2=0x55aaca2080a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.233+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca205cb0 0x55aaca2080a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.233+0000 7f67d6ffd640 1 -- 172.21.15.46:0/3149369737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1faaf0 msgr2=0x55aaca1faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.233+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1faaf0 0x55aaca1faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.233+0000 7f67d6ffd640 1 -- 172.21.15.46:0/3149369737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaca106340 con 0x55aaca1fb410 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.236+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67d000f470 tx=0x7f67d000f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33afbe1c5f16420e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.236+0000 7f67d77fe640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 0x7f67c00a5490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0b8380 tx=0x7f67cc005b50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.240+0000 7f67bdffb640 1 -- 172.21.15.46:0/3149369737 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004d20 con 0x55aaca1fb410 2024-09-12T03:01:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.240+0000 7f67bdffb640 1 -- 172.21.15.46:0/3149369737 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d0010c10 con 0x55aaca1fb410 2024-09-12T03:01:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.240+0000 7f67bdffb640 1 -- 172.21.15.46:0/3149369737 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00058e0 con 0x55aaca1fb410 2024-09-12T03:01:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.242+0000 7f67e0800740 1 -- 172.21.15.46:0/3149369737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 msgr2=0x55aaca205770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.242+0000 7f67e0800740 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67d000f470 tx=0x7f67d000f930 comp rx=0 tx=0).stop 2024-09-12T03:01:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.245+0000 7f67e0800740 1 -- 172.21.15.46:0/3149369737 shutdown_connections 2024-09-12T03:01:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.245+0000 7f67e0800740 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca205cb0 0x55aaca2080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.245+0000 7f67e0800740 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca205770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.245+0000 7f67e0800740 1 --2- 172.21.15.46:0/3149369737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1faaf0 0x55aaca1faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.245+0000 7f67e0800740 1 -- 172.21.15.46:0/3149369737 >> 172.21.15.46:0/3149369737 conn(0x55aaca1f7060 msgr2=0x55aaca1f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.250+0000 7f67e0800740 1 -- 172.21.15.46:0/3149369737 shutdown_connections 2024-09-12T03:01:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.252+0000 7f67e0800740 1 -- 172.21.15.46:0/3149369737 wait complete. 2024-09-12T03:01:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.252+0000 7f67e0800740 1 -- start start 2024-09-12T03:01:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.253+0000 7f67e0800740 1 Processor -- start 2024-09-12T03:01:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca202c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca2036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aaca208880 con 0x55aaca1fb410 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aaca0e8290 con 0x55aaca205cb0 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aaca208a00 con 0x55aaca1faaf0 2024-09-12T03:01:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.257+0000 7f67e0800740 10 client.0 _ll_register_callbacks cb 0x55aaca119140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35706/0 (socket says 172.21.15.46:35706) 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d6ffd640 1 -- 172.21.15.46:0/3121471128 learned_addr learned my addr 172.21.15.46:0/3121471128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca2036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca202c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 -- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 msgr2=0x55aaca202c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca202c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 -- 172.21.15.46:0/3121471128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 msgr2=0x55aaca203170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 -- 172.21.15.46:0/3121471128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aac9e72210 con 0x55aaca205cb0 2024-09-12T03:01:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.260+0000 7f67d67fc640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca2036b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005d80 tx=0x7f67c800d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763ea204bb0525fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.263+0000 7f67b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:01:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.264+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c80108a0 con 0x55aaca205cb0 2024-09-12T03:01:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.264+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9555+0+0 (secure 0 0 0) 0x7f67c8010a40 con 0x55aaca205cb0 2024-09-12T03:01:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.267+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d0010c10 con 0x55aaca205cb0 2024-09-12T03:01:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.273+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f67c8022050 con 0x55aaca205cb0 2024-09-12T03:01:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.276+0000 7f67e0800740 10 client.25757 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.276+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aac9f3e620 con 0x55aaca205cb0 2024-09-12T03:01:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d0010c10 con 0x55aaca205cb0 2024-09-12T03:01:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f67c801dc30 con 0x55aaca205cb0 2024-09-12T03:01:14.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:13 smithi046 ceph-mon[30951]: pgmap v344: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:01:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 10 client.25757 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 20 client.25757 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca216190 0x55aaca2185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 4 client.25757 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.280+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaca16ca20 con 0x55aaca216190 2024-09-12T03:01:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.282+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca216190 0x55aaca2185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.289+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca216190 0x55aaca2185f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d872a54b3bd63d server_cookie=3a4ca8a194742248 in_seq=0 out_seq=0 2024-09-12T03:01:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.289+0000 7f67b6ffd640 10 client.25757 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.293+0000 7f67b6ffd640 1 -- 172.21.15.46:0/3121471128 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaca16ca20 con 0x55aaca216190 2024-09-12T03:01:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.293+0000 7f67b6ffd640 10 client.25757 handle_command_reply: tid=0 2024-09-12T03:01:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.293+0000 7f67e0800740 1 client.25757 shutdown 2024-09-12T03:01:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.298+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 msgr2=0x55aaca2036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.298+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca2036b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005d80 tx=0x7f67c800d2e0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 shutdown_connections 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca216190 0x55aaca2185f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca2036b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 --2- 172.21.15.46:0/3121471128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca202c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.302+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 >> 172.21.15.46:0/3121471128 conn(0x55aaca1f7060 msgr2=0x55aaca1fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.305+0000 7f67b6ffd640 0 client.25757 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.306+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 shutdown_connections 2024-09-12T03:01:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.307+0000 7f67e0800740 1 -- 172.21.15.46:0/3121471128 wait complete. 2024-09-12T03:01:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.307+0000 7f67e0800740 20 client.25757 trim_cache size 0 max 16384 2024-09-12T03:01:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 Processor -- start 2024-09-12T03:01:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 -- start start 2024-09-12T03:01:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca1fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca21dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81da030 con 0x55aaca1fb410 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8084d60 con 0x55aaca205cb0 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.376+0000 7f67e0800740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8115c40 con 0x55aaca1faaf0 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35708/0 (socket says 172.21.15.46:35708) 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2278166564 learned_addr learned my addr 172.21.15.46:0/2278166564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2278166564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 msgr2=0x55aaca1fd560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca1fd560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2278166564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 msgr2=0x55aaca21dec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca21dec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.377+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2278166564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaca212660 con 0x55aaca1fb410 2024-09-12T03:01:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.378+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005530 tx=0x7f67d0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9224a9bb81766426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.383+0000 7f67bd7fa640 1 -- 172.21.15.46:0/2278166564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d001d040 con 0x55aaca1fb410 2024-09-12T03:01:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.383+0000 7f67bd7fa640 1 -- 172.21.15.46:0/2278166564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d0010c10 con 0x55aaca1fb410 2024-09-12T03:01:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.383+0000 7f67bd7fa640 1 -- 172.21.15.46:0/2278166564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004d60 con 0x55aaca1fb410 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.385+0000 7f67e0800740 1 -- 172.21.15.46:0/2278166564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 msgr2=0x55aaca203520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.385+0000 7f67e0800740 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005530 tx=0x7f67d0007f70 comp rx=0 tx=0).stop 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.386+0000 7f67e0800740 1 -- 172.21.15.46:0/2278166564 shutdown_connections 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.386+0000 7f67e0800740 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca205cb0 0x55aaca21dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.386+0000 7f67e0800740 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca1fb410 0x55aaca203520 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.386+0000 7f67e0800740 1 --2- 172.21.15.46:0/2278166564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca1fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.386+0000 7f67e0800740 1 -- 172.21.15.46:0/2278166564 >> 172.21.15.46:0/2278166564 conn(0x55aaca1f7060 msgr2=0x55aaca201d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.393+0000 7f67e0800740 1 -- 172.21.15.46:0/2278166564 shutdown_connections 2024-09-12T03:01:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.398+0000 7f67e0800740 1 -- 172.21.15.46:0/2278166564 wait complete. 2024-09-12T03:01:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 -- start start 2024-09-12T03:01:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 Processor -- start 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca2009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca20dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca205cb0 0x55aaca20e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8083240 con 0x55aaca205cb0 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8084690 con 0x55aaca1fb410 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80838f0 con 0x55aaca1faaf0 2024-09-12T03:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.399+0000 7f67e0800740 10 client.0 _ll_register_callbacks cb 0x7f67c800d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.400+0000 7f67d67fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca2009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.400+0000 7f67d67fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca2009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:49678/0 (socket says 172.21.15.46:49678) 2024-09-12T03:01:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.400+0000 7f67d67fc640 1 -- 172.21.15.46:0/2252964964 learned_addr learned my addr 172.21.15.46:0/2252964964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.400+0000 7f67d67fc640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca205cb0 0x55aaca20e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.400+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca20dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.403+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 msgr2=0x55aaca2009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.403+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca2009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.403+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2252964964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca205cb0 msgr2=0x55aaca20e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.403+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca205cb0 0x55aaca20e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.403+0000 7f67d6ffd640 1 -- 172.21.15.46:0/2252964964 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaca1fef80 con 0x55aaca1fb410 2024-09-12T03:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.405+0000 7f67d6ffd640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca20dc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55aaca205b10 tx=0x7f67d00053d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a3641f717a7d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.405+0000 7f67b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.406+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0010760 con 0x55aaca1fb410 2024-09-12T03:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.406+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9555+0+0 (secure 0 0 0) 0x7f67d0010900 con 0x55aaca1fb410 2024-09-12T03:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.412+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d0010c10 con 0x55aaca1fb410 2024-09-12T03:01:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.416+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f67d0005560 con 0x55aaca1fb410 2024-09-12T03:01:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.424+0000 7f67e0800740 10 client.25760 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.424+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aaca280220 con 0x55aaca1fb410 2024-09-12T03:01:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d0010c10 con 0x55aaca1fb410 2024-09-12T03:01:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f67d002a2f0 con 0x55aaca1fb410 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 10 client.25760 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 20 client.25760 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca280830 0x55aaca282c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 4 client.25760 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.428+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaca166d70 con 0x55aaca280830 2024-09-12T03:01:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.429+0000 7f67d67fc640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca280830 0x55aaca282c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.436+0000 7f67d67fc640 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca280830 0x55aaca282c50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd23112f9e14375 server_cookie=d47ec684cc3edbe7 in_seq=0 out_seq=0 2024-09-12T03:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.437+0000 7f67b6ffd640 10 client.25760 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.824+0000 7f67b6ffd640 1 -- 172.21.15.46:0/2252964964 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaca166d70 con 0x55aaca280830 2024-09-12T03:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.824+0000 7f67b6ffd640 10 client.25760 handle_command_reply: tid=0 2024-09-12T03:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.824+0000 7f67e0800740 1 client.25760 shutdown 2024-09-12T03:01:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.832+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 msgr2=0x55aaca20dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.832+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca20dc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55aaca205b10 tx=0x7f67d00053d0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.833+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 shutdown_connections 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.833+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55aaca280830 0x55aaca282c50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.833+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aaca205cb0 0x55aaca20e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.833+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaca1fb410 0x55aaca20dc90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.834+0000 7f67e0800740 1 --2- 172.21.15.46:0/2252964964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaca1faaf0 0x55aaca2009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.834+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 >> 172.21.15.46:0/2252964964 conn(0x55aaca1f7060 msgr2=0x55aaca204490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.834+0000 7f67b6ffd640 0 client.25760 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.837+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 shutdown_connections 2024-09-12T03:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.837+0000 7f67e0800740 1 -- 172.21.15.46:0/2252964964 wait complete. 2024-09-12T03:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.838+0000 7f67e0800740 20 client.25760 trim_cache size 0 max 16384 2024-09-12T03:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.844+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 msgr2=0x7f67c00a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.844+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 0x7f67c00a5490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0b8380 tx=0x7f67cc005b50 comp rx=0 tx=0).stop 2024-09-12T03:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.844+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 msgr2=0x7f67d8114920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.844+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d8114920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009850 tx=0x7f67cc003cd0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 shutdown_connections 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f67c00a2fd0 0x7f67c00a5490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67d811b140 0x7f67d8114920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67d81095d0 0x7f67d811abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 --2- 172.21.15.46:0/2332758284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d81091f0 0x7f67d811a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.851+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 >> 172.21.15.46:0/2332758284 conn(0x7f67d806ce00 msgr2=0x7f67d80707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.857+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 shutdown_connections 2024-09-12T03:01:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:14.860+0000 7f67dd98d640 1 -- 172.21.15.46:0/2332758284 wait complete. 2024-09-12T03:01:16.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:15 smithi145 ceph-mon[34674]: pgmap v345: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:01:16.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:15 smithi196 ceph-mon[34587]: pgmap v345: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:01:16.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:15 smithi046 ceph-mon[30951]: pgmap v345: 129 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-12T03:01:18.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:17 smithi145 ceph-mon[34674]: pgmap v346: 129 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:01:18.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:17 smithi196 ceph-mon[34587]: pgmap v346: 129 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:01:18.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:17 smithi046 ceph-mon[30951]: pgmap v346: 129 pgs: 10 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-12T03:01:20.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:19 smithi145 ceph-mon[34674]: pgmap v347: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:01:20.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:19 smithi196 ceph-mon[34587]: pgmap v347: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:01:20.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:19 smithi046 ceph-mon[30951]: pgmap v347: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-09-12T03:01:21.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:21 smithi145 ceph-mon[34674]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:01:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:21 smithi196 ceph-mon[34587]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:01:21.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:21 smithi046 ceph-mon[30951]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:01:22.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:22 smithi145 ceph-mon[34674]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2024-09-12T03:01:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:22 smithi196 ceph-mon[34587]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2024-09-12T03:01:22.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:22 smithi046 ceph-mon[30951]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2024-09-12T03:01:24.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:24 smithi145 ceph-mon[34674]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:24 smithi196 ceph-mon[34587]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:24 smithi046 ceph-mon[30951]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:26.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:26 smithi145 ceph-mon[34674]: pgmap v351: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:26 smithi196 ceph-mon[34587]: pgmap v351: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:26 smithi046 ceph-mon[30951]: pgmap v351: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.092+0000 7f1a3705e640 1 -- 172.21.15.46:0/815719352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301091f0 msgr2=0x7f1a30110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.092+0000 7f1a3705e640 1 --2- 172.21.15.46:0/815719352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301091f0 0x7f1a30110250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009900 tx=0x7f1a2002a440 comp rx=0 tx=0).stop 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.097+0000 7f1a3705e640 1 -- 172.21.15.46:0/815719352 shutdown_connections 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.097+0000 7f1a3705e640 1 --2- 172.21.15.46:0/815719352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a3011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.097+0000 7f1a3705e640 1 --2- 172.21.15.46:0/815719352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301095d0 0x7f1a30109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.097+0000 7f1a3705e640 1 --2- 172.21.15.46:0/815719352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301091f0 0x7f1a30110250 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.097+0000 7f1a3705e640 1 -- 172.21.15.46:0/815719352 >> 172.21.15.46:0/815719352 conn(0x7f1a3006cc60 msgr2=0x7f1a3006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.098+0000 7f1a3705e640 1 -- 172.21.15.46:0/815719352 shutdown_connections 2024-09-12T03:01:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.099+0000 7f1a3705e640 1 -- 172.21.15.46:0/815719352 wait complete. 2024-09-12T03:01:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 Processor -- start 2024-09-12T03:01:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 -- start start 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301091f0 0x7f1a301187a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301095d0 0x7f1a30111820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30120a50 con 0x7f1a301095d0 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301208d0 con 0x7f1a3011b140 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.100+0000 7f1a3705e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30120bd0 con 0x7f1a301091f0 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.102+0000 7f1a355fb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.102+0000 7f1a355fb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:44182/0 (socket says 172.21.15.46:44182) 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.102+0000 7f1a355fb640 1 -- 172.21.15.46:0/2218518971 learned_addr learned my addr 172.21.15.46:0/2218518971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.103+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301091f0 0x7f1a301187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.105+0000 7f1a355fb640 1 -- 172.21.15.46:0/2218518971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301091f0 msgr2=0x7f1a301187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.105+0000 7f1a355fb640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301091f0 0x7f1a301187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.106+0000 7f1a355fb640 1 -- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301095d0 msgr2=0x7f1a30111820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:01:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.106+0000 7f1a355fb640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301095d0 0x7f1a30111820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.106+0000 7f1a355fb640 1 -- 172.21.15.46:0/2218518971 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301123d0 con 0x7f1a3011b140 2024-09-12T03:01:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.108+0000 7f1a355fb640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800b9e0 tx=0x7f1a2800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.118+0000 7f1a2dffb640 1 -- 172.21.15.46:0/2218518971 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28019070 con 0x7f1a3011b140 2024-09-12T03:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.118+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301162c0 con 0x7f1a3011b140 2024-09-12T03:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.118+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a30116810 con 0x7f1a3011b140 2024-09-12T03:01:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.122+0000 7f1a39dfd740 1 Processor -- start 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 -- start start 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa96230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa962d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 0x556aa9630270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa95872f0 con 0x556aa96235e0 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556aa952f270 con 0x556aa9622cc0 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.123+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556aa95832b0 con 0x556aa962de80 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.124+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa96230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.124+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 0x556aa9630270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.124+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa96230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:44196/0 (socket says 172.21.15.46:44196) 2024-09-12T03:01:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.124+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2125910074 learned_addr learned my addr 172.21.15.46:0/2125910074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.125+0000 7f1a2dffb640 1 -- 172.21.15.46:0/2218518971 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a28014440 con 0x7f1a3011b140 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.125+0000 7f1a2dffb640 1 -- 172.21.15.46:0/2218518971 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280125a0 con 0x7f1a3011b140 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.127+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2125910074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 msgr2=0x556aa96230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.127+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa96230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.127+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2125910074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 msgr2=0x556aa962d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.127+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa962d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.127+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2125910074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa952e340 con 0x556aa962de80 2024-09-12T03:01:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.128+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 0x556aa9630270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a18013750 tx=0x7f1a18005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b31ebaa6806bc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.128+0000 7f1a26ffd640 1 -- 172.21.15.46:0/2125910074 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18004220 con 0x556aa962de80 2024-09-12T03:01:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.128+0000 7f1a26ffd640 1 -- 172.21.15.46:0/2125910074 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a180043c0 con 0x556aa962de80 2024-09-12T03:01:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.128+0000 7f1a26ffd640 1 -- 172.21.15.46:0/2125910074 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18003bb0 con 0x556aa962de80 2024-09-12T03:01:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.135+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2125910074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 msgr2=0x556aa9630270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.135+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 0x556aa9630270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a18013750 tx=0x7f1a18005f90 comp rx=0 tx=0).stop 2024-09-12T03:01:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.140+0000 7f1a2dffb640 1 -- 172.21.15.46:0/2218518971 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1a2801c070 con 0x7f1a3011b140 2024-09-12T03:01:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.141+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2125910074 shutdown_connections 2024-09-12T03:01:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.141+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa962de80 0x556aa9630270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.141+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa962d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.141+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2125910074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa96230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.141+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2125910074 >> 172.21.15.46:0/2125910074 conn(0x556aa961f230 msgr2=0x556aa961f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.144+0000 7f1a2dffb640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 0x7f1a1c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.144+0000 7f1a2dffb640 1 -- 172.21.15.46:0/2218518971 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7f1a280ceca0 con 0x7f1a3011b140 2024-09-12T03:01:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.144+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2125910074 shutdown_connections 2024-09-12T03:01:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.145+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 0x7f1a1c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.146+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2125910074 wait complete. 2024-09-12T03:01:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.146+0000 7f1a39dfd740 1 -- start start 2024-09-12T03:01:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.146+0000 7f1a39dfd740 1 Processor -- start 2024-09-12T03:01:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa962ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa962b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556aa9630a50 con 0x556aa962de80 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556aa9510290 con 0x556aa9622cc0 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556aa9630bd0 con 0x556aa96235e0 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 0x7f1a1c0a53c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x556aa9620090 tx=0x7f1a20000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44434/0 (socket says 172.21.15.46:44434) 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a2ffff640 1 -- 172.21.15.46:0/1812395509 learned_addr learned my addr 172.21.15.46:0/1812395509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.147+0000 7f1a39dfd740 10 client.0 _ll_register_callbacks cb 0x556aa9541140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.148+0000 7f1a355fb640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa962b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.153+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa962ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.154+0000 7f1a355fb640 1 -- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 msgr2=0x556aa962b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.154+0000 7f1a355fb640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.154+0000 7f1a355fb640 1 -- 172.21.15.46:0/1812395509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 msgr2=0x556aa962ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.154+0000 7f1a355fb640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa962ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.154+0000 7f1a355fb640 1 -- 172.21.15.46:0/1812395509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa929a210 con 0x556aa962de80 2024-09-12T03:01:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.159+0000 7f1a355fb640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa962b860 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1a280cc0e0 tx=0x7f1a2809b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3e9b19b179a2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.160+0000 7f1a0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:01:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.160+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2809f260 con 0x556aa962de80 2024-09-12T03:01:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.161+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a180043c0 con 0x556aa962de80 2024-09-12T03:01:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.161+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7f1a280145f0 con 0x556aa962de80 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.161+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a280a3b20 con 0x556aa962de80 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.161+0000 7f1a39dfd740 10 client.16191 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.161+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aa9366620 con 0x556aa962de80 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.162+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a180043c0 con 0x556aa962de80 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1a280b9af0 con 0x556aa962de80 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 10 client.16191 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 20 client.16191 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa963e290 0x556aa9640710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 4 client.16191 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.163+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa9594a20 con 0x556aa963e290 2024-09-12T03:01:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.165+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa963e290 0x556aa9640710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.169+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa963e290 0x556aa9640710 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8f31f4bdf52dbe server_cookie=61ed5ab8440a0de1 in_seq=0 out_seq=0 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.169+0000 7f1a0ffff640 10 client.16191 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.172+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1812395509 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556aa9594a20 con 0x556aa963e290 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.172+0000 7f1a0ffff640 10 client.16191 handle_command_reply: tid=0 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.172+0000 7f1a39dfd740 1 client.16191 shutdown 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.173+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 msgr2=0x556aa962b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.173+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa962b860 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1a280cc0e0 tx=0x7f1a2809b5b0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 shutdown_connections 2024-09-12T03:01:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa963e290 0x556aa9640710 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa962b860 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1812395509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa962ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.181+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 >> 172.21.15.46:0/1812395509 conn(0x556aa961f230 msgr2=0x556aa9624f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.184+0000 7f1a0ffff640 0 client.16191 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.188+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 shutdown_connections 2024-09-12T03:01:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.188+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1812395509 wait complete. 2024-09-12T03:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.189+0000 7f1a39dfd740 20 client.16191 trim_cache size 0 max 16384 2024-09-12T03:01:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.243+0000 7f1a39dfd740 1 Processor -- start 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 -- start start 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa9625780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa961e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a34dfa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa9625780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a34dfa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa9625780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:44208/0 (socket says 172.21.15.46:44208) 2024-09-12T03:01:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a34dfa640 1 -- 172.21.15.46:0/2027872534 learned_addr learned my addr 172.21.15.46:0/2027872534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301d9d50 con 0x556aa962de80 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b55c0 con 0x556aa9622cc0 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.248+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30109a10 con 0x556aa96235e0 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.250+0000 7f1a355fb640 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa961e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2027872534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 msgr2=0x556aa9625780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa9625780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2027872534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 msgr2=0x556aa961e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa961e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.251+0000 7f1a2ffff640 1 -- 172.21.15.46:0/2027872534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa9629330 con 0x556aa96235e0 2024-09-12T03:01:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.252+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x556aa962c420 tx=0x7f1a1800efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401f47e58ad68136 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.253+0000 7f1a267fc640 1 -- 172.21.15.46:0/2027872534 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800eb60 con 0x556aa96235e0 2024-09-12T03:01:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.253+0000 7f1a267fc640 1 -- 172.21.15.46:0/2027872534 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a18013280 con 0x556aa96235e0 2024-09-12T03:01:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.253+0000 7f1a267fc640 1 -- 172.21.15.46:0/2027872534 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011e00 con 0x556aa96235e0 2024-09-12T03:01:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.255+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 msgr2=0x556aa962b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.255+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x556aa962c420 tx=0x7f1a1800efd0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.263+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 shutdown_connections 2024-09-12T03:01:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.263+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa962de80 0x556aa961e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.263+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa96235e0 0x556aa962b690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.263+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/2027872534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa9622cc0 0x556aa9625780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.263+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 >> 172.21.15.46:0/2027872534 conn(0x556aa961f230 msgr2=0x556aa961da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.268+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 shutdown_connections 2024-09-12T03:01:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.268+0000 7f1a39dfd740 1 -- 172.21.15.46:0/2027872534 wait complete. 2024-09-12T03:01:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.268+0000 7f1a39dfd740 1 -- start start 2024-09-12T03:01:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.269+0000 7f1a39dfd740 1 Processor -- start 2024-09-12T03:01:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.269+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a39dfd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa9636690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a34dfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a34dfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44454/0 (socket says 172.21.15.46:44454) 2024-09-12T03:01:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a34dfa640 1 -- 172.21.15.46:0/1911038273 learned_addr learned my addr 172.21.15.46:0/1911038273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa962de80 0x556aa9627850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.270+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30117000 con 0x556aa96235e0 2024-09-12T03:01:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.271+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b4ef0 con 0x556aa962de80 2024-09-12T03:01:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.271+0000 7f1a355fb640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa962de80 0x556aa9627850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.271+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b4150 con 0x556aa9622cc0 2024-09-12T03:01:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.271+0000 7f1a39dfd740 10 client.0 _ll_register_callbacks cb 0x7f1a28014960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa9636690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a34dfa640 1 -- 172.21.15.46:0/1911038273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa962de80 msgr2=0x556aa9627850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa962de80 0x556aa9627850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a34dfa640 1 -- 172.21.15.46:0/1911038273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 msgr2=0x556aa9636690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa9636690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.276+0000 7f1a34dfa640 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa9628080 con 0x556aa9622cc0 2024-09-12T03:01:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.277+0000 7f1a34dfa640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009230 tx=0x7f1a20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c433e384c46feb8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.277+0000 7f1a0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:01:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.279+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20031d60 con 0x556aa9622cc0 2024-09-12T03:01:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.280+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a18013280 con 0x556aa9622cc0 2024-09-12T03:01:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.280+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7f1a2004b430 con 0x556aa9622cc0 2024-09-12T03:01:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.284+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a20063050 con 0x556aa9622cc0 2024-09-12T03:01:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.284+0000 7f1a39dfd740 10 client.25741 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.284+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aa96a8c60 con 0x556aa9622cc0 2024-09-12T03:01:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.288+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a18013280 con 0x556aa9622cc0 2024-09-12T03:01:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1a2003d6f0 con 0x556aa9622cc0 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 10 client.25741 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 20 client.25741 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa96a94a0 0x556aa96ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 4 client.25741 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:01:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.289+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aa958ed70 con 0x556aa96a94a0 2024-09-12T03:01:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.291+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa96a94a0 0x556aa96ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.296+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa96a94a0 0x556aa96ab8c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c37676b0202fe9 server_cookie=10773fb6c8136271 in_seq=0 out_seq=0 2024-09-12T03:01:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.296+0000 7f1a0ffff640 10 client.25741 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:28.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:28 smithi145 ceph-mon[34674]: pgmap v352: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:01:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.688+0000 7f1a0ffff640 1 -- 172.21.15.46:0/1911038273 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556aa958ed70 con 0x556aa96a94a0 2024-09-12T03:01:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.688+0000 7f1a0ffff640 10 client.25741 handle_command_reply: tid=0 2024-09-12T03:01:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.688+0000 7f1a39dfd740 1 client.25741 shutdown 2024-09-12T03:01:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.688+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 msgr2=0x556aa9636150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.688+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009230 tx=0x7f1a20005fd0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 shutdown_connections 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556aa96a94a0 0x556aa96ab8c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556aa962de80 0x556aa9627850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556aa96235e0 0x556aa9636690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 --2- 172.21.15.46:0/1911038273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aa9622cc0 0x556aa9636150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 >> 172.21.15.46:0/1911038273 conn(0x556aa961f230 msgr2=0x556aa961e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a0ffff640 0 client.25741 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 shutdown_connections 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 1 -- 172.21.15.46:0/1911038273 wait complete. 2024-09-12T03:01:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.689+0000 7f1a39dfd740 20 client.25741 trim_cache size 0 max 16384 2024-09-12T03:01:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:01:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:01:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:01:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:01:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.692+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 msgr2=0x7f1a1c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.693+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 0x7f1a1c0a53c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x556aa9620090 tx=0x7f1a20000bc0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.693+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 msgr2=0x7f1a30111d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.693+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800b9e0 tx=0x7f1a2800bea0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.697+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 shutdown_connections 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.697+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1a1c0a2f00 0x7f1a1c0a53c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.697+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a3011b140 0x7f1a30111d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.698+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301095d0 0x7f1a30111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.698+0000 7f1a3705e640 1 --2- 172.21.15.46:0/2218518971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a301091f0 0x7f1a301187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.698+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 >> 172.21.15.46:0/2218518971 conn(0x7f1a3006cc60 msgr2=0x7f1a3011b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.699+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 shutdown_connections 2024-09-12T03:01:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:28.699+0000 7f1a3705e640 1 -- 172.21.15.46:0/2218518971 wait complete. 2024-09-12T03:01:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:28 smithi196 ceph-mon[34587]: pgmap v352: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:01:28.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:28 smithi046 ceph-mon[30951]: pgmap v352: 129 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 105 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:01:30.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:30 smithi145 ceph-mon[34674]: pgmap v353: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:30 smithi196 ceph-mon[34587]: pgmap v353: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:30.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:30 smithi046 ceph-mon[30951]: pgmap v353: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.91k op/s 2024-09-12T03:01:32.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:32 smithi145 ceph-mon[34674]: pgmap v354: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2024-09-12T03:01:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:32 smithi196 ceph-mon[34587]: pgmap v354: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2024-09-12T03:01:32.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:32 smithi046 ceph-mon[30951]: pgmap v354: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2024-09-12T03:01:34.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:34 smithi145 ceph-mon[34674]: pgmap v355: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-09-12T03:01:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:34 smithi196 ceph-mon[34587]: pgmap v355: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-09-12T03:01:34.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:34 smithi046 ceph-mon[30951]: pgmap v355: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-09-12T03:01:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:36 smithi196 ceph-mon[34587]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-12T03:01:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:36 smithi046 ceph-mon[30951]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-12T03:01:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:36 smithi145 ceph-mon[34674]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 115 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-12T03:01:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:38 smithi196 ceph-mon[34587]: pgmap v357: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2024-09-12T03:01:38.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:38 smithi046 ceph-mon[30951]: pgmap v357: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2024-09-12T03:01:38.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:38 smithi145 ceph-mon[34674]: pgmap v357: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2024-09-12T03:01:39.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:39.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:39 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:39.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:39 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:39.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:39 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:40 smithi196 ceph-mon[34587]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T03:01:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:40 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:40 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:40 smithi046 ceph-mon[30951]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T03:01:40.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:40 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:40 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:40 smithi145 ceph-mon[34674]: pgmap v358: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-12T03:01:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:40 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:40 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:41 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:41 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:41.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:41 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:41.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:41 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:41.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:41 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:41.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:41 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:41 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:41 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:41.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:41 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:01:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.015+0000 7fd525d13640 1 -- 172.21.15.46:0/2373377965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 msgr2=0x7fd520110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.015+0000 7fd525d13640 1 --2- 172.21.15.46:0/2373377965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd520110250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd51402a400 comp rx=0 tx=0).stop 2024-09-12T03:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.018+0000 7fd525d13640 1 -- 172.21.15.46:0/2373377965 shutdown_connections 2024-09-12T03:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.018+0000 7fd525d13640 1 --2- 172.21.15.46:0/2373377965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52011b140 0x7fd52011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.018+0000 7fd525d13640 1 --2- 172.21.15.46:0/2373377965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5201095d0 0x7fd520109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.018+0000 7fd525d13640 1 --2- 172.21.15.46:0/2373377965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd520110250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.018+0000 7fd525d13640 1 -- 172.21.15.46:0/2373377965 >> 172.21.15.46:0/2373377965 conn(0x7fd52006cc60 msgr2=0x7fd52006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.023+0000 7fd525d13640 1 -- 172.21.15.46:0/2373377965 shutdown_connections 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.023+0000 7fd525d13640 1 -- 172.21.15.46:0/2373377965 wait complete. 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 Processor -- start 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 -- start start 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd5201a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5201095d0 0x7fd5201a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd520120a50 con 0x7fd5201095d0 2024-09-12T03:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201208d0 con 0x7fd5201091f0 2024-09-12T03:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.024+0000 7fd525d13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd520120bd0 con 0x7fd52011b140 2024-09-12T03:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46540/0 (socket says 172.21.15.46:46540) 2024-09-12T03:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 -- 172.21.15.46:0/2627854816 learned_addr learned my addr 172.21.15.46:0/2627854816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51effd640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5201095d0 0x7fd5201a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd5201a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 -- 172.21.15.46:0/2627854816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 msgr2=0x7fd5201a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd5201a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 -- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5201095d0 msgr2=0x7fd5201a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5201095d0 0x7fd5201a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.025+0000 7fd51ffff640 1 -- 172.21.15.46:0/2627854816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5201abff0 con 0x7fd52011b140 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.026+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd5201a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.026+0000 7fd51ffff640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b9e0 tx=0x7fd51800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.027+0000 7fd51cff9640 1 -- 172.21.15.46:0/2627854816 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518019070 con 0x7fd52011b140 2024-09-12T03:01:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.027+0000 7fd51cff9640 1 -- 172.21.15.46:0/2627854816 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd518014440 con 0x7fd52011b140 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.027+0000 7fd51cff9640 1 -- 172.21.15.46:0/2627854816 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180125a0 con 0x7fd52011b140 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.030+0000 7fd525d13640 1 -- 172.21.15.46:0/2627854816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5201ac2e0 con 0x7fd52011b140 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.030+0000 7fd525d13640 1 -- 172.21.15.46:0/2627854816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52007a450 con 0x7fd52011b140 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.033+0000 7fd528aa3740 1 Processor -- start 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.036+0000 7fd51cff9640 1 -- 172.21.15.46:0/2627854816 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd5180145f0 con 0x7fd52011b140 2024-09-12T03:01:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.037+0000 7fd51cff9640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 0x7fd5080a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.038+0000 7fd51cff9640 1 -- 172.21.15.46:0/2627854816 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7fd5180ceb30 con 0x7fd52011b140 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 -- start start 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d01e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d02780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556351c552f0 con 0x556351d03640 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556351bfd270 con 0x556351d0cc90 2024-09-12T03:01:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.040+0000 7fd528aa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556351c512b0 con 0x556351d02340 2024-09-12T03:01:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.042+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.042+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:41106/0 (socket says 172.21.15.46:41106) 2024-09-12T03:01:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.042+0000 7fd51ffff640 1 -- 172.21.15.46:0/3377771315 learned_addr learned my addr 172.21.15.46:0/3377771315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.044+0000 7fd51effd640 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d02780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.044+0000 7fd51effd640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 0x7fd5080a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.044+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d01e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.049+0000 7fd51ffff640 1 -- 172.21.15.46:0/3377771315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 msgr2=0x556351d02780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.049+0000 7fd51ffff640 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d02780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.049+0000 7fd51ffff640 1 -- 172.21.15.46:0/3377771315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 msgr2=0x556351d01e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.049+0000 7fd51ffff640 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d01e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.050+0000 7fd51ffff640 1 -- 172.21.15.46:0/3377771315 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556351bfc340 con 0x556351d0cc90 2024-09-12T03:01:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.050+0000 7fd51ffff640 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b9b0 tx=0x7fd51809adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3699da26031b3c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.053+0000 7fd525d13640 1 -- 172.21.15.46:0/3377771315 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180149a0 con 0x556351d0cc90 2024-09-12T03:01:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.053+0000 7fd525d13640 1 -- 172.21.15.46:0/3377771315 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd518097460 con 0x556351d0cc90 2024-09-12T03:01:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.053+0000 7fd525d13640 1 -- 172.21.15.46:0/3377771315 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a3760 con 0x556351d0cc90 2024-09-12T03:01:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.054+0000 7fd528aa3740 1 -- 172.21.15.46:0/3377771315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 msgr2=0x556351d0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.054+0000 7fd528aa3740 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b9b0 tx=0x7fd51809adf0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.059+0000 7fd51effd640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 0x7fd5080a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd52011ca10 tx=0x7fd51000a760 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.067+0000 7fd528aa3740 1 -- 172.21.15.46:0/3377771315 shutdown_connections 2024-09-12T03:01:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.067+0000 7fd528aa3740 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d0f080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.067+0000 7fd528aa3740 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d02780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.067+0000 7fd528aa3740 1 --2- 172.21.15.46:0/3377771315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d01e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.067+0000 7fd528aa3740 1 -- 172.21.15.46:0/3377771315 >> 172.21.15.46:0/3377771315 conn(0x556351cfe580 msgr2=0x556351cfe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.074+0000 7fd528aa3740 1 -- 172.21.15.46:0/3377771315 shutdown_connections 2024-09-12T03:01:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.079+0000 7fd528aa3740 1 -- 172.21.15.46:0/3377771315 wait complete. 2024-09-12T03:01:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 -- start start 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 Processor -- start 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d02340 0x556351d09ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 0x556351d0a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 0x556351d0a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556351d0fb20 con 0x556351d0cc90 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556351bde290 con 0x556351d02340 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556351d0fca0 con 0x556351d03640 2024-09-12T03:01:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.080+0000 7fd528aa3740 10 client.0 _ll_register_callbacks cb 0x556351c0f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.081+0000 7fd51effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 0x556351d0a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 0x556351d0a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46556/0 (socket says 172.21.15.46:46556) 2024-09-12T03:01:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51effd640 1 -- 172.21.15.46:0/1464650252 learned_addr learned my addr 172.21.15.46:0/1464650252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 0x556351d0a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 -- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 msgr2=0x556351d0a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 0x556351d0a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 -- 172.21.15.46:0/1464650252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d02340 msgr2=0x556351d09ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d02340 0x556351d09ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.082+0000 7fd51ffff640 1 -- 172.21.15.46:0/1464650252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556351968210 con 0x556351d0cc90 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.083+0000 7fd51ffff640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 0x556351d0a720 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5180977c0 tx=0x7fd518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d638b34cf3ad4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.083+0000 7fd4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.083+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a8060 con 0x556351d0cc90 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.083+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd518097460 con 0x556351d0cc90 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.083+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7fd51809a240 con 0x556351d0cc90 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.084+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd51809a4c0 con 0x556351d0cc90 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.084+0000 7fd528aa3740 10 client.16206 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.084+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556351a34620 con 0x556351d0cc90 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.084+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd518097460 con 0x556351d0cc90 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.084+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd518099cb0 con 0x556351d0cc90 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 10 client.16206 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 20 client.16206 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d1d0c0 0x556351d1f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 4 client.16206 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.086+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556351c62a20 con 0x556351d1d0c0 2024-09-12T03:01:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.090+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d1d0c0 0x556351d1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.096+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d1d0c0 0x556351d1f560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620ab1e750224e95 server_cookie=3c81dd0cf1a84e1f in_seq=0 out_seq=0 2024-09-12T03:01:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.096+0000 7fd4feffd640 10 client.16206 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.099+0000 7fd4feffd640 1 -- 172.21.15.46:0/1464650252 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556351c62a20 con 0x556351d1d0c0 2024-09-12T03:01:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.099+0000 7fd4feffd640 10 client.16206 handle_command_reply: tid=0 2024-09-12T03:01:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.099+0000 7fd528aa3740 1 client.16206 shutdown 2024-09-12T03:01:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.101+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 msgr2=0x556351d0a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.101+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 0x556351d0a720 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5180977c0 tx=0x7fd518004a40 comp rx=0 tx=0).stop 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 shutdown_connections 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d1d0c0 0x556351d1f560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d0cc90 0x556351d0a720 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d03640 0x556351d0a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 --2- 172.21.15.46:0/1464650252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d02340 0x556351d09ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.105+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 >> 172.21.15.46:0/1464650252 conn(0x556351cfe580 msgr2=0x556351cec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.109+0000 7fd4feffd640 0 client.16206 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.118+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 shutdown_connections 2024-09-12T03:01:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.118+0000 7fd528aa3740 1 -- 172.21.15.46:0/1464650252 wait complete. 2024-09-12T03:01:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.120+0000 7fd528aa3740 20 client.16206 trim_cache size 0 max 16384 2024-09-12T03:01:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.174+0000 7fd528aa3740 1 Processor -- start 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 -- start start 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351cec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d24e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d25370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd51f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351cec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd51f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351cec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:46560/0 (socket says 172.21.15.46:46560) 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd51f7fe640 1 -- 172.21.15.46:0/496963753 learned_addr learned my addr 172.21.15.46:0/496963753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd52007e820 con 0x556351d03640 2024-09-12T03:01:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd52007c060 con 0x556351d0cc90 2024-09-12T03:01:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.176+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5201188d0 con 0x556351d02340 2024-09-12T03:01:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.177+0000 7fd51effd640 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d24e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.179+0000 7fd51ffff640 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d25370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 -- 172.21.15.46:0/496963753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 msgr2=0x556351cec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351cec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 -- 172.21.15.46:0/496963753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 msgr2=0x556351d25370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d25370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 -- 172.21.15.46:0/496963753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556351d02040 con 0x556351d03640 2024-09-12T03:01:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.181+0000 7fd51effd640 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d24e30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x556351d0b2e0 tx=0x7fd51001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6f392bb8708ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.182+0000 7fd5057fa640 1 -- 172.21.15.46:0/496963753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000fba0 con 0x556351d03640 2024-09-12T03:01:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.182+0000 7fd5057fa640 1 -- 172.21.15.46:0/496963753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd51000fd40 con 0x556351d03640 2024-09-12T03:01:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.183+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 msgr2=0x556351d24e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.183+0000 7fd528aa3740 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d24e30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x556351d0b2e0 tx=0x7fd51001e000 comp rx=0 tx=0).stop 2024-09-12T03:01:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.187+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 shutdown_connections 2024-09-12T03:01:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.187+0000 7fd528aa3740 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d25370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.187+0000 7fd528aa3740 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d24e30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.187+0000 7fd528aa3740 1 --2- 172.21.15.46:0/496963753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351cec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.187+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 >> 172.21.15.46:0/496963753 conn(0x556351cfe580 msgr2=0x556351cebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.188+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 shutdown_connections 2024-09-12T03:01:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.197+0000 7fd528aa3740 1 -- 172.21.15.46:0/496963753 wait complete. 2024-09-12T03:01:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.197+0000 7fd528aa3740 1 -- start start 2024-09-12T03:01:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.197+0000 7fd528aa3740 1 Processor -- start 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d15470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd520118f40 con 0x556351d03640 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd52007b990 con 0x556351d0cc90 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd52007abf0 con 0x556351d02340 2024-09-12T03:01:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.200+0000 7fd528aa3740 10 client.0 _ll_register_callbacks cb 0x7fd514009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.201+0000 7fd51effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d15470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.201+0000 7fd51effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d15470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55936/0 (socket says 172.21.15.46:55936) 2024-09-12T03:01:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.201+0000 7fd51effd640 1 -- 172.21.15.46:0/319469738 learned_addr learned my addr 172.21.15.46:0/319469738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.201+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.202+0000 7fd51ffff640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.207+0000 7fd51f7fe640 1 -- 172.21.15.46:0/319469738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 msgr2=0x556351d062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.207+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.207+0000 7fd51f7fe640 1 -- 172.21.15.46:0/319469738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 msgr2=0x556351d15470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.207+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d15470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.207+0000 7fd51f7fe640 1 -- 172.21.15.46:0/319469738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556351d06800 con 0x556351d02340 2024-09-12T03:01:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.209+0000 7fd51f7fe640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d14f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5140096c0 tx=0x7fd5140045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3126ad75fff23bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.209+0000 7fd4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:01:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514035040 con 0x556351d02340 2024-09-12T03:01:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556351c5cd70 con 0x556351d02340 2024-09-12T03:01:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9479+0+0 (secure 0 0 0) 0x7fd514005590 con 0x556351d02340 2024-09-12T03:01:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5140046c0 con 0x556351d02340 2024-09-12T03:01:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd528aa3740 10 client.25756 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.210+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556351d87720 con 0x556351d02340 2024-09-12T03:01:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.211+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556351c5cd70 con 0x556351d02340 2024-09-12T03:01:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd514003b90 con 0x556351d02340 2024-09-12T03:01:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 10 client.25756 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 20 client.25756 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d87fa0 0x556351d8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 4 client.25756 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.212+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556351bd9900 con 0x556351d87fa0 2024-09-12T03:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.213+0000 7fd51effd640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d87fa0 0x556351d8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.216+0000 7fd51effd640 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d87fa0 0x556351d8a3c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d8e2d5f4c6769e server_cookie=7d5402043752ab9a in_seq=0 out_seq=0 2024-09-12T03:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.217+0000 7fd4feffd640 10 client.25756 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.376+0000 7fd4feffd640 1 -- 172.21.15.46:0/319469738 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556351bd9900 con 0x556351d87fa0 2024-09-12T03:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.376+0000 7fd4feffd640 10 client.25756 handle_command_reply: tid=0 2024-09-12T03:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.376+0000 7fd528aa3740 1 client.25756 shutdown 2024-09-12T03:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.379+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 msgr2=0x556351d14f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.379+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d14f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5140096c0 tx=0x7fd5140045a0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 shutdown_connections 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x556351d87fa0 0x556351d8a3c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556351d0cc90 0x556351d062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556351d03640 0x556351d15470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 --2- 172.21.15.46:0/319469738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556351d02340 0x556351d14f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 >> 172.21.15.46:0/319469738 conn(0x556351cfe580 msgr2=0x556351d042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.389+0000 7fd4feffd640 0 client.25756 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.392+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 shutdown_connections 2024-09-12T03:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.395+0000 7fd528aa3740 1 -- 172.21.15.46:0/319469738 wait complete. 2024-09-12T03:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.396+0000 7fd528aa3740 20 client.25756 trim_cache size 0 max 16384 2024-09-12T03:01:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:01:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:01:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:01:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:01:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.416+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 msgr2=0x7fd5080a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.416+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 0x7fd5080a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd52011ca10 tx=0x7fd51000a760 comp rx=0 tx=0).stop 2024-09-12T03:01:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.416+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 msgr2=0x7fd5201ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.416+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b9e0 tx=0x7fd51800bea0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 shutdown_connections 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd5080a2f00 0x7fd5080a53c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52011b140 0x7fd5201ab8c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5201095d0 0x7fd5201a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 --2- 172.21.15.46:0/2627854816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5201091f0 0x7fd5201a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.427+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 >> 172.21.15.46:0/2627854816 conn(0x7fd52006cc60 msgr2=0x7fd52011b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.429+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 shutdown_connections 2024-09-12T03:01:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:42.430+0000 7fd5067fc640 1 -- 172.21.15.46:0/2627854816 wait complete. 2024-09-12T03:01:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:42 smithi196 ceph-mon[34587]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-12T03:01:42.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:42 smithi046 ceph-mon[30951]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-12T03:01:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:42 smithi145 ceph-mon[34674]: pgmap v359: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-12T03:01:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:43 smithi196 ceph-mon[34587]: pgmap v360: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-09-12T03:01:43.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:43 smithi046 ceph-mon[30951]: pgmap v360: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-09-12T03:01:43.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:43 smithi145 ceph-mon[34674]: pgmap v360: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-09-12T03:01:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:45 smithi196 ceph-mon[34587]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-12T03:01:46.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:45 smithi046 ceph-mon[30951]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-12T03:01:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:45 smithi145 ceph-mon[34674]: pgmap v361: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-12T03:01:47.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:47 smithi196 ceph-mon[34587]: pgmap v362: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:01:48.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:47 smithi046 ceph-mon[30951]: pgmap v362: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:01:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:47 smithi145 ceph-mon[34674]: pgmap v362: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:01:50.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:49 smithi046 ceph-mon[30951]: pgmap v363: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:50.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:49 smithi145 ceph-mon[34674]: pgmap v363: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:49 smithi196 ceph-mon[34587]: pgmap v363: 129 pgs: 5 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:01:52.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:52 smithi145 ceph-mon[34674]: pgmap v364: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-12T03:01:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:52 smithi196 ceph-mon[34587]: pgmap v364: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-12T03:01:52.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:52 smithi046 ceph-mon[30951]: pgmap v364: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-12T03:01:54.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:54 smithi145 ceph-mon[34674]: pgmap v365: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2024-09-12T03:01:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:54 smithi196 ceph-mon[34587]: pgmap v365: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2024-09-12T03:01:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:54 smithi046 ceph-mon[30951]: pgmap v365: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2024-09-12T03:01:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.652+0000 7fc33be0e640 1 -- 172.21.15.46:0/3724121451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3341095d0 msgr2=0x7fc334109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.652+0000 7fc33be0e640 1 --2- 172.21.15.46:0/3724121451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3341095d0 0x7fc334109a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009c10 tx=0x7fc32c02a700 comp rx=0 tx=0).stop 2024-09-12T03:01:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.659+0000 7fc33be0e640 1 -- 172.21.15.46:0/3724121451 shutdown_connections 2024-09-12T03:01:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.659+0000 7fc33be0e640 1 --2- 172.21.15.46:0/3724121451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc33411b140 0x7fc33411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.659+0000 7fc33be0e640 1 --2- 172.21.15.46:0/3724121451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3341095d0 0x7fc334109a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.659+0000 7fc33be0e640 1 --2- 172.21.15.46:0/3724121451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341091f0 0x7fc334110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.659+0000 7fc33be0e640 1 -- 172.21.15.46:0/3724121451 >> 172.21.15.46:0/3724121451 conn(0x7fc33406ce00 msgr2=0x7fc33406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.665+0000 7fc33be0e640 1 -- 172.21.15.46:0/3724121451 shutdown_connections 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.667+0000 7fc33be0e640 1 -- 172.21.15.46:0/3724121451 wait complete. 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 Processor -- start 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 -- start start 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3341091f0 0x7fc334118720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 0x7fc3341117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 0x7fc334111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc334120ba0 con 0x7fc3341095d0 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc334120a20 con 0x7fc33411b140 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc33be0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc334120d20 con 0x7fc3341091f0 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 0x7fc3341117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 0x7fc3341117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47328/0 (socket says 172.21.15.46:47328) 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.668+0000 7fc3393a9640 1 -- 172.21.15.46:0/1833668350 learned_addr learned my addr 172.21.15.46:0/1833668350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.672+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 0x7fc334111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.674+0000 7fc33a3ab640 1 -- 172.21.15.46:0/1833668350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3341091f0 msgr2=0x7fc334118720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.674+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3341091f0 0x7fc334118720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.674+0000 7fc33a3ab640 1 -- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 msgr2=0x7fc3341117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.674+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 0x7fc3341117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.674+0000 7fc33a3ab640 1 -- 172.21.15.46:0/1833668350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3341122d0 con 0x7fc33411b140 2024-09-12T03:01:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.678+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 0x7fc334111ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e970 tx=0x7fc32800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.682+0000 7fc332ffd640 1 -- 172.21.15.46:0/1833668350 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32800ccc0 con 0x7fc33411b140 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.682+0000 7fc332ffd640 1 -- 172.21.15.46:0/1833668350 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc328005020 con 0x7fc33411b140 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.682+0000 7fc332ffd640 1 -- 172.21.15.46:0/1833668350 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328004a00 con 0x7fc33411b140 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.684+0000 7fc33be0e640 1 -- 172.21.15.46:0/1833668350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341125c0 con 0x7fc33411b140 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.684+0000 7fc33be0e640 1 -- 172.21.15.46:0/1833668350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc334116570 con 0x7fc33411b140 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 Processor -- start 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 -- start start 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652cdc1c2f0 con 0x5652cdcc2a90 2024-09-12T03:01:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5652cdbc4270 con 0x5652cdcb8140 2024-09-12T03:01:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.688+0000 7fc33ec83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5652cdc182b0 con 0x5652cdcb9440 2024-09-12T03:01:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.689+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.689+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44520/0 (socket says 172.21.15.46:44520) 2024-09-12T03:01:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.689+0000 7fc3393a9640 1 -- 172.21.15.46:0/3782440566 learned_addr learned my addr 172.21.15.46:0/3782440566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.692+0000 7fc339baa640 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.693+0000 7fc3393a9640 1 -- 172.21.15.46:0/3782440566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 msgr2=0x5652cdcb8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.693+0000 7fc3393a9640 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcb8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.693+0000 7fc3393a9640 1 -- 172.21.15.46:0/3782440566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 msgr2=0x5652cdcc4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.693+0000 7fc3393a9640 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.693+0000 7fc3393a9640 1 -- 172.21.15.46:0/3782440566 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cdbc3340 con 0x5652cdcb9440 2024-09-12T03:01:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.702+0000 7fc3393a9640 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009be0 tx=0x7fc32c030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649086718694a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.703+0000 7fc332ffd640 1 -- 172.21.15.46:0/1833668350 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc328004ba0 con 0x7fc33411b140 2024-09-12T03:01:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.704+0000 7fc332ffd640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 0x7fc31c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.704+0000 7fc33be0e640 1 -- 172.21.15.46:0/3782440566 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c036070 con 0x5652cdcb9440 2024-09-12T03:01:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.704+0000 7fc33be0e640 1 -- 172.21.15.46:0/3782440566 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc32c044070 con 0x5652cdcb9440 2024-09-12T03:01:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.705+0000 7fc332ffd640 1 -- 172.21.15.46:0/1833668350 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9451+0+0 (secure 0 0 0) 0x7fc3280997d0 con 0x7fc33411b140 2024-09-12T03:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.705+0000 7fc33ec83740 1 -- 172.21.15.46:0/3782440566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 msgr2=0x5652cdcb7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.705+0000 7fc33ec83740 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009be0 tx=0x7fc32c030430 comp rx=0 tx=0).stop 2024-09-12T03:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.709+0000 7fc339baa640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 0x7fc31c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.710+0000 7fc339baa640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 0x7fc31c0a5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5652cdcb51e0 tx=0x7fc324009260 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.712+0000 7fc33ec83740 1 -- 172.21.15.46:0/3782440566 shutdown_connections 2024-09-12T03:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.712+0000 7fc33ec83740 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.712+0000 7fc33ec83740 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.712+0000 7fc33ec83740 1 --2- 172.21.15.46:0/3782440566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcb7c00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.712+0000 7fc33ec83740 1 -- 172.21.15.46:0/3782440566 >> 172.21.15.46:0/3782440566 conn(0x5652cdcb4380 msgr2=0x5652cdcb4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.718+0000 7fc33ec83740 1 -- 172.21.15.46:0/3782440566 shutdown_connections 2024-09-12T03:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.718+0000 7fc33ec83740 1 -- 172.21.15.46:0/3782440566 wait complete. 2024-09-12T03:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.718+0000 7fc33ec83740 1 -- start start 2024-09-12T03:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.718+0000 7fc33ec83740 1 Processor -- start 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcbfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652cdcc5910 con 0x5652cdcc2a90 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5652cdba5290 con 0x5652cdcb8140 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5652cdcc5a90 con 0x5652cdcb9440 2024-09-12T03:01:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.719+0000 7fc33ec83740 10 client.0 _ll_register_callbacks cb 0x5652cdbd6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.720+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.720+0000 7fc3393a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44526/0 (socket says 172.21.15.46:44526) 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.720+0000 7fc3393a9640 1 -- 172.21.15.46:0/96171195 learned_addr learned my addr 172.21.15.46:0/96171195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.720+0000 7fc339baa640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcbfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 -- 172.21.15.46:0/96171195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 msgr2=0x5652cdcbfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcbfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 -- 172.21.15.46:0/96171195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 msgr2=0x5652cdcc04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 -- 172.21.15.46:0/96171195 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cd92f210 con 0x5652cdcb9440 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3393a9640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc32c006010 tx=0x7fc32c004720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9024d2ae82769157 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.721+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c036050 con 0x5652cdcb9440 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.722+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652cdc29a20 con 0x5652cdcb9440 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.722+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9451+0+0 (secure 0 0 0) 0x7fc32c030d10 con 0x5652cdcb9440 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.722+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc32c05b050 con 0x5652cdcb9440 2024-09-12T03:01:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.722+0000 7fc33ec83740 10 client.25765 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.722+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652cd9fb620 con 0x5652cdcb9440 2024-09-12T03:01:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.723+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5652cdc29a20 con 0x5652cdcb9440 2024-09-12T03:01:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fc32c03c070 con 0x5652cdcb9440 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 10 client.25765 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 20 client.25765 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdcd3100 0x5652cdcd5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 4 client.25765 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.724+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652cdc23d70 con 0x5652cdcd3100 2024-09-12T03:01:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.725+0000 7fc339baa640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdcd3100 0x5652cdcd5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.730+0000 7fc339baa640 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdcd3100 0x5652cdcd5560 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203b36179dcc4da7 server_cookie=9209f9205160063d in_seq=0 out_seq=0 2024-09-12T03:01:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.730+0000 7fc3197fa640 10 client.25765 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.733+0000 7fc3197fa640 1 -- 172.21.15.46:0/96171195 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652cdc23d70 con 0x5652cdcd3100 2024-09-12T03:01:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.733+0000 7fc3197fa640 10 client.25765 handle_command_reply: tid=0 2024-09-12T03:01:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 client.25765 shutdown 2024-09-12T03:01:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 msgr2=0x5652cdcbff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc32c006010 tx=0x7fc32c004720 comp rx=0 tx=0).stop 2024-09-12T03:01:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 shutdown_connections 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdcd3100 0x5652cdcd5560 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcc2a90 0x5652cdcc04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb9440 0x5652cdcbff60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 --2- 172.21.15.46:0/96171195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdcbfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.736+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 >> 172.21.15.46:0/96171195 conn(0x5652cdcb4380 msgr2=0x5652cdcb3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.737+0000 7fc3197fa640 0 client.25765 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.737+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 shutdown_connections 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.737+0000 7fc33ec83740 1 -- 172.21.15.46:0/96171195 wait complete. 2024-09-12T03:01:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.737+0000 7fc33ec83740 20 client.25765 trim_cache size 0 max 16384 2024-09-12T03:01:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.801+0000 7fc33ec83740 1 Processor -- start 2024-09-12T03:01:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.804+0000 7fc33ec83740 1 -- start start 2024-09-12T03:01:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 0x5652cdcc02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcdae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc339baa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 0x5652cdcc02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc339baa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 0x5652cdcc02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:44528/0 (socket says 172.21.15.46:44528) 2024-09-12T03:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc339baa640 1 -- 172.21.15.46:0/2636687194 learned_addr learned my addr 172.21.15.46:0/2636687194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.805+0000 7fc33ec83740 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 0x5652cdcdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.806+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc320012720 con 0x5652cdcb9440 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.806+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc32000ff60 con 0x5652cdcc2a90 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.806+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc32000de60 con 0x5652cdcb8140 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.806+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 0x5652cdcdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc3393a9640 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcdae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc33a3ab640 1 -- 172.21.15.46:0/2636687194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 msgr2=0x5652cdcc02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 0x5652cdcc02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc33a3ab640 1 -- 172.21.15.46:0/2636687194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 msgr2=0x5652cdcdae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcdae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.807+0000 7fc33a3ab640 1 -- 172.21.15.46:0/2636687194 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cdcbdf50 con 0x5652cdcc2a90 2024-09-12T03:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.808+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 0x5652cdcdb370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cc960 tx=0x7fc3280a2df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca32419c2a1d2afc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.809+0000 7fc31b7fe640 1 -- 172.21.15.46:0/2636687194 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328098160 con 0x5652cdcc2a90 2024-09-12T03:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.809+0000 7fc31b7fe640 1 -- 172.21.15.46:0/2636687194 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc328015e80 con 0x5652cdcc2a90 2024-09-12T03:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.810+0000 7fc31b7fe640 1 -- 172.21.15.46:0/2636687194 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328098300 con 0x5652cdcc2a90 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 msgr2=0x5652cdcdb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 0x5652cdcdb370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cc960 tx=0x7fc3280a2df0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 shutdown_connections 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcc2a90 0x5652cdcdb370 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcdae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 --2- 172.21.15.46:0/2636687194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcb8140 0x5652cdcc02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.811+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 >> 172.21.15.46:0/2636687194 conn(0x5652cdcb4380 msgr2=0x5652cdcbe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.835+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 shutdown_connections 2024-09-12T03:01:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.835+0000 7fc33ec83740 1 -- 172.21.15.46:0/2636687194 wait complete. 2024-09-12T03:01:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.836+0000 7fc33ec83740 1 -- start start 2024-09-12T03:01:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.836+0000 7fc33ec83740 1 Processor -- start 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcc5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcc2a90 0x5652cdcc6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc32000e420 con 0x5652cdcb9440 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc32000f890 con 0x5652cdcb8140 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc32000eaf0 con 0x5652cdcc2a90 2024-09-12T03:01:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.837+0000 7fc33ec83740 10 client.0 _ll_register_callbacks cb 0x7fc328005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:48686/0 (socket says 172.21.15.46:48686) 2024-09-12T03:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 -- 172.21.15.46:0/892941417 learned_addr learned my addr 172.21.15.46:0/892941417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc33a3ab640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcc2a90 0x5652cdcc6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 -- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcc2a90 msgr2=0x5652cdcc6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcc2a90 0x5652cdcc6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 -- 172.21.15.46:0/892941417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 msgr2=0x5652cdcc5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcc5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.838+0000 7fc339baa640 1 -- 172.21.15.46:0/892941417 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cdcc6b00 con 0x5652cdcb8140 2024-09-12T03:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.839+0000 7fc339baa640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc32401f550 tx=0x7fc324004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38942c8fada34ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.841+0000 7fc3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.841+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324004320 con 0x5652cdcb8140 2024-09-12T03:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.841+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc328015e80 con 0x5652cdcb8140 2024-09-12T03:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.841+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9451+0+0 (secure 0 0 0) 0x7fc3240035f0 con 0x5652cdcb8140 2024-09-12T03:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.841+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc324039050 con 0x5652cdcb8140 2024-09-12T03:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.847+0000 7fc33ec83740 10 client.25802 fetch_fsmap learned FSMap version 35 2024-09-12T03:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.847+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652cdd3ce30 con 0x5652cdcb8140 2024-09-12T03:01:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc328015e80 con 0x5652cdcb8140 2024-09-12T03:01:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fc324028810 con 0x5652cdcb8140 2024-09-12T03:01:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 10 client.25802 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 20 client.25802 populate_metadata read hostname 'smithi046' 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdd3d5f0 0x5652cdd3fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 4 client.25802 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652cdba0900 con 0x5652cdd3d5f0 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.852+0000 7fc3393a9640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdd3d5f0 0x5652cdd3fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.853+0000 7fc3197fa640 10 client.25802 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:55.854+0000 7fc3393a9640 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdd3d5f0 0x5652cdd3fa30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6eea62e00085c21 server_cookie=44ed69d4e9c9a239 in_seq=0 out_seq=0 2024-09-12T03:01:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.061+0000 7fc3197fa640 1 -- 172.21.15.46:0/892941417 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652cdba0900 con 0x5652cdd3d5f0 2024-09-12T03:01:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.061+0000 7fc3197fa640 10 client.25802 handle_command_reply: tid=0 2024-09-12T03:01:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.061+0000 7fc33ec83740 1 client.25802 shutdown 2024-09-12T03:01:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.064+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 msgr2=0x5652cdccb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.064+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc32401f550 tx=0x7fc324004730 comp rx=0 tx=0).stop 2024-09-12T03:01:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 shutdown_connections 2024-09-12T03:01:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5652cdd3d5f0 0x5652cdd3fa30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cdcc2a90 0x5652cdcc6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652cdcb9440 0x5652cdcc5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 --2- 172.21.15.46:0/892941417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5652cdcb8140 0x5652cdccb4d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 >> 172.21.15.46:0/892941417 conn(0x5652cdcb4380 msgr2=0x5652cdcb2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc3197fa640 0 client.25802 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 shutdown_connections 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 1 -- 172.21.15.46:0/892941417 wait complete. 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.068+0000 7fc33ec83740 20 client.25802 trim_cache size 0 max 16384 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:01:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:01:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:01:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.071+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 msgr2=0x7fc31c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.071+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 0x7fc31c0a5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5652cdcb51e0 tx=0x7fc324009260 comp rx=0 tx=0).stop 2024-09-12T03:01:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.071+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 msgr2=0x7fc334111ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.071+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 0x7fc334111ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e970 tx=0x7fc32800ee30 comp rx=0 tx=0).stop 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 shutdown_connections 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc31c0a2f50 0x7fc31c0a5410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc33411b140 0x7fc334111ce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3341095d0 0x7fc3341117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 --2- 172.21.15.46:0/1833668350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3341091f0 0x7fc334118720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.074+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 >> 172.21.15.46:0/1833668350 conn(0x7fc33406ce00 msgr2=0x7fc33411b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.075+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 shutdown_connections 2024-09-12T03:01:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:01:56.077+0000 7fc330bff640 1 -- 172.21.15.46:0/1833668350 wait complete. 2024-09-12T03:01:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:56 smithi145 ceph-mon[34674]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:01:56.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:56 smithi145 ceph-mon[34674]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:01:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:56 smithi196 ceph-mon[34587]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:01:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:56 smithi196 ceph-mon[34587]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:01:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:56 smithi046 ceph-mon[30951]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:01:56.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:56 smithi046 ceph-mon[30951]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:01:58.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:01:58 smithi145 ceph-mon[34674]: pgmap v368: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-12T03:01:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:01:58 smithi196 ceph-mon[34587]: pgmap v368: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-12T03:01:58.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:01:58 smithi046 ceph-mon[30951]: pgmap v368: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-12T03:02:00.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:00 smithi145 ceph-mon[34674]: pgmap v369: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.04k op/s 2024-09-12T03:02:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:00 smithi196 ceph-mon[34587]: pgmap v369: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.04k op/s 2024-09-12T03:02:00.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:00 smithi046 ceph-mon[30951]: pgmap v369: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.04k op/s 2024-09-12T03:02:02.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:02 smithi145 ceph-mon[34674]: pgmap v370: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-12T03:02:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:02 smithi196 ceph-mon[34587]: pgmap v370: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-12T03:02:02.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:02 smithi046 ceph-mon[30951]: pgmap v370: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-09-12T03:02:04.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:04 smithi145 ceph-mon[34674]: pgmap v371: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:04 smithi196 ceph-mon[34587]: pgmap v371: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:04.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:04 smithi046 ceph-mon[30951]: pgmap v371: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:06 smithi145 ceph-mon[34674]: pgmap v372: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-12T03:02:06.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:06 smithi145 ceph-mon[34674]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:02:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:06 smithi196 ceph-mon[34587]: pgmap v372: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-12T03:02:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:06 smithi196 ceph-mon[34587]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:02:06.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:06 smithi046 ceph-mon[30951]: pgmap v372: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2024-09-12T03:02:06.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:06 smithi046 ceph-mon[30951]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:02:08.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:08 smithi145 ceph-mon[34674]: pgmap v374: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:08 smithi196 ceph-mon[34587]: pgmap v374: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:08.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:08 smithi046 ceph-mon[30951]: pgmap v374: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-09-12T03:02:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.322+0000 7f109a59d640 1 -- 172.21.15.46:0/232738982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10941095d0 msgr2=0x7f1094109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.322+0000 7f109a59d640 1 --2- 172.21.15.46:0/232738982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10941095d0 0x7f1094109a30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1090009860 tx=0x7f109002a360 comp rx=0 tx=0).stop 2024-09-12T03:02:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.332+0000 7f109a59d640 1 -- 172.21.15.46:0/232738982 shutdown_connections 2024-09-12T03:02:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.332+0000 7f109a59d640 1 --2- 172.21.15.46:0/232738982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f109411b140 0x7f109411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.332+0000 7f109a59d640 1 --2- 172.21.15.46:0/232738982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10941095d0 0x7f1094109a30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.332+0000 7f109a59d640 1 --2- 172.21.15.46:0/232738982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 0x7f1094110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.332+0000 7f109a59d640 1 -- 172.21.15.46:0/232738982 >> 172.21.15.46:0/232738982 conn(0x7f109406cbf0 msgr2=0x7f109406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.335+0000 7f109a59d640 1 -- 172.21.15.46:0/232738982 shutdown_connections 2024-09-12T03:02:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.341+0000 7f109a59d640 1 -- 172.21.15.46:0/232738982 wait complete. 2024-09-12T03:02:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.342+0000 7f109a59d640 1 Processor -- start 2024-09-12T03:02:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 -- start start 2024-09-12T03:02:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 0x7f10941a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941095d0 0x7f10941a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10941209c0 con 0x7f109411b140 2024-09-12T03:02:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1094120840 con 0x7f10941091f0 2024-09-12T03:02:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.343+0000 7f109a59d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1094120b40 con 0x7f10941095d0 2024-09-12T03:02:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.344+0000 7f1099d9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.344+0000 7f1099d9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41608/0 (socket says 172.21.15.46:41608) 2024-09-12T03:02:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.344+0000 7f1099d9c640 1 -- 172.21.15.46:0/4182916127 learned_addr learned my addr 172.21.15.46:0/4182916127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.345+0000 7f109959b640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 0x7f10941a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.346+0000 7f1098d9a640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941095d0 0x7f10941a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.349+0000 7f1099d9c640 1 -- 172.21.15.46:0/4182916127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941095d0 msgr2=0x7f10941a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.349+0000 7f1099d9c640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941095d0 0x7f10941a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.349+0000 7f1099d9c640 1 -- 172.21.15.46:0/4182916127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 msgr2=0x7f10941a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.349+0000 7f1099d9c640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 0x7f10941a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.349+0000 7f1099d9c640 1 -- 172.21.15.46:0/4182916127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10941abf20 con 0x7f109411b140 2024-09-12T03:02:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.350+0000 7f1099d9c640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f108c009fd0 tx=0x7f108c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.354+0000 7f108a7fc640 1 -- 172.21.15.46:0/4182916127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004b20 con 0x7f109411b140 2024-09-12T03:02:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.354+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10941ac210 con 0x7f109411b140 2024-09-12T03:02:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.354+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1094110920 con 0x7f109411b140 2024-09-12T03:02:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.354+0000 7f108a7fc640 1 -- 172.21.15.46:0/4182916127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f108c003e20 con 0x7f109411b140 2024-09-12T03:02:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.354+0000 7f108a7fc640 1 -- 172.21.15.46:0/4182916127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004810 con 0x7f109411b140 2024-09-12T03:02:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.361+0000 7f108a7fc640 1 -- 172.21.15.46:0/4182916127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f108c01d020 con 0x7f109411b140 2024-09-12T03:02:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.362+0000 7f108a7fc640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 0x7f10800a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.363+0000 7f108a7fc640 1 -- 172.21.15.46:0/4182916127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f108c0d0590 con 0x7f109411b140 2024-09-12T03:02:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.363+0000 7f109959b640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 0x7f10800a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.364+0000 7f109959b640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 0x7f10800a5560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10840058c0 tx=0x7f108400a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 Processor -- start 2024-09-12T03:02:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 -- start start 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eefe4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefe7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616eef3e2f0 con 0x5616eefe4e80 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5616eeee6270 con 0x5616eefda5e0 2024-09-12T03:02:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.365+0000 7f109e66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616eef3a2b0 con 0x5616eefd9cc0 2024-09-12T03:02:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.367+0000 7f1098d9a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eefe4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.367+0000 7f109959b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.367+0000 7f1098d9a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eefe4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:49822/0 (socket says 172.21.15.46:49822) 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.367+0000 7f1098d9a640 1 -- 172.21.15.46:0/1292176865 learned_addr learned my addr 172.21.15.46:0/1292176865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f1099d9c640 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefe7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f109959b640 1 -- 172.21.15.46:0/1292176865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 msgr2=0x5616eefe4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f109959b640 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eefe4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f109959b640 1 -- 172.21.15.46:0/1292176865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 msgr2=0x5616eefe7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f109959b640 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefe7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.368+0000 7f109959b640 1 -- 172.21.15.46:0/1292176865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616eeee5340 con 0x5616eefd9cc0 2024-09-12T03:02:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.373+0000 7f109959b640 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefda0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f108400a940 tx=0x7f1084012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b9ac33f6ce2f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.380+0000 7f107f7fe640 1 -- 172.21.15.46:0/1292176865 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084016070 con 0x5616eefd9cc0 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.380+0000 7f107f7fe640 1 -- 172.21.15.46:0/1292176865 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1084010e40 con 0x5616eefd9cc0 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.380+0000 7f107f7fe640 1 -- 172.21.15.46:0/1292176865 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108401c660 con 0x5616eefd9cc0 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 -- 172.21.15.46:0/1292176865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 msgr2=0x5616eefda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefda0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f108400a940 tx=0x7f1084012000 comp rx=0 tx=0).stop 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 -- 172.21.15.46:0/1292176865 shutdown_connections 2024-09-12T03:02:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefe7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eefe4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 --2- 172.21.15.46:0/1292176865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefda0a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.383+0000 7f109e66e740 1 -- 172.21.15.46:0/1292176865 >> 172.21.15.46:0/1292176865 conn(0x5616eefd6230 msgr2=0x5616eefd6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.389+0000 7f109e66e740 1 -- 172.21.15.46:0/1292176865 shutdown_connections 2024-09-12T03:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.392+0000 7f109e66e740 1 -- 172.21.15.46:0/1292176865 wait complete. 2024-09-12T03:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.392+0000 7f109e66e740 1 -- start start 2024-09-12T03:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.392+0000 7f109e66e740 1 Processor -- start 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefe1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 0x5616eefe2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 0x5616eefe27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616eefe7a50 con 0x5616eefda5e0 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5616eeec7290 con 0x5616eefe4e80 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.399+0000 7f109e66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616eefe7bd0 con 0x5616eefd9cc0 2024-09-12T03:02:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.400+0000 7f109e66e740 10 client.0 _ll_register_callbacks cb 0x5616eeef8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.401+0000 7f1099d9c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 0x5616eefe27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.401+0000 7f1099d9c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 0x5616eefe27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:49828/0 (socket says 172.21.15.46:49828) 2024-09-12T03:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.401+0000 7f1099d9c640 1 -- 172.21.15.46:0/1932912048 learned_addr learned my addr 172.21.15.46:0/1932912048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.401+0000 7f109959b640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefe1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.402+0000 7f1098d9a640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 0x5616eefe2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 -- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 msgr2=0x5616eefe1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefe1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 -- 172.21.15.46:0/1932912048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 msgr2=0x5616eefe27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 0x5616eefe27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 -- 172.21.15.46:0/1932912048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616eec51210 con 0x5616eefda5e0 2024-09-12T03:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f1098d9a640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 0x5616eefe2260 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5616eefd7090 tx=0x7f1090004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb14f5e9ce1ae3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.403+0000 7f107cff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:02:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.405+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090042070 con 0x5616eefda5e0 2024-09-12T03:02:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.405+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616eef4ba20 con 0x5616eefda5e0 2024-09-12T03:02:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.405+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f1090004470 con 0x5616eefda5e0 2024-09-12T03:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.407+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f109004e050 con 0x5616eefda5e0 2024-09-12T03:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.407+0000 7f109e66e740 10 client.16233 fetch_fsmap learned FSMap version 35 2024-09-12T03:02:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.407+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5616eed1d620 con 0x5616eefda5e0 2024-09-12T03:02:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5616eef4ba20 con 0x5616eefda5e0 2024-09-12T03:02:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1090003d20 con 0x5616eefda5e0 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 10 client.16233 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 20 client.16233 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616eeff5240 0x5616eeff7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 4 client.16233 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616eef45d70 con 0x5616eeff5240 2024-09-12T03:02:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.409+0000 7f109959b640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616eeff5240 0x5616eeff7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.411+0000 7f107cff9640 10 client.16233 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.412+0000 7f109959b640 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616eeff5240 0x5616eeff7680 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5d1c75627e307d server_cookie=7cc54cb18748c809 in_seq=0 out_seq=0 2024-09-12T03:02:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.416+0000 7f107cff9640 1 -- 172.21.15.46:0/1932912048 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616eef45d70 con 0x5616eeff5240 2024-09-12T03:02:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.416+0000 7f107cff9640 10 client.16233 handle_command_reply: tid=0 2024-09-12T03:02:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.416+0000 7f109e66e740 1 client.16233 shutdown 2024-09-12T03:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.421+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 msgr2=0x5616eefe2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.421+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 0x5616eefe2260 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5616eefd7090 tx=0x7f1090004040 comp rx=0 tx=0).stop 2024-09-12T03:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 shutdown_connections 2024-09-12T03:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616eeff5240 0x5616eeff7680 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefe4e80 0x5616eefe27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefda5e0 0x5616eefe2260 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 --2- 172.21.15.46:0/1932912048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefd9cc0 0x5616eefe1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.424+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 >> 172.21.15.46:0/1932912048 conn(0x5616eefd6230 msgr2=0x5616eefdbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.425+0000 7f107cff9640 0 client.16233 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.425+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 shutdown_connections 2024-09-12T03:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.426+0000 7f109e66e740 1 -- 172.21.15.46:0/1932912048 wait complete. 2024-09-12T03:02:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.426+0000 7f109e66e740 20 client.16233 trim_cache size 0 max 16384 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.467+0000 7f109e66e740 1 Processor -- start 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.467+0000 7f109e66e740 1 -- start start 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eeffcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefe4e80 0x5616eeffd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10780128e0 con 0x5616eefd9cc0 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1078010120 con 0x5616eefda5e0 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109e66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f107800df90 con 0x5616eefe4e80 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109959b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109959b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41628/0 (socket says 172.21.15.46:41628) 2024-09-12T03:02:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f109959b640 1 -- 172.21.15.46:0/2672087212 learned_addr learned my addr 172.21.15.46:0/2672087212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f1099d9c640 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefe4e80 0x5616eeffd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.468+0000 7f1098d9a640 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eeffcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 -- 172.21.15.46:0/2672087212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefe4e80 msgr2=0x5616eeffd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefe4e80 0x5616eeffd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 -- 172.21.15.46:0/2672087212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 msgr2=0x5616eeffcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eeffcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 -- 172.21.15.46:0/2672087212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616eefe0270 con 0x5616eefd9cc0 2024-09-12T03:02:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.469+0000 7f109959b640 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f108400f790 tx=0x7f108401bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06980835b45e731 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.474+0000 7f107effd640 1 -- 172.21.15.46:0/2672087212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084016070 con 0x5616eefd9cc0 2024-09-12T03:02:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.474+0000 7f107effd640 1 -- 172.21.15.46:0/2672087212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1084035070 con 0x5616eefd9cc0 2024-09-12T03:02:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.474+0000 7f107effd640 1 -- 172.21.15.46:0/2672087212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108402a040 con 0x5616eefd9cc0 2024-09-12T03:02:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.475+0000 7f109e66e740 1 -- 172.21.15.46:0/2672087212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 msgr2=0x5616eefe25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.475+0000 7f109e66e740 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f108400f790 tx=0x7f108401bed0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.483+0000 7f109e66e740 1 -- 172.21.15.46:0/2672087212 shutdown_connections 2024-09-12T03:02:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.483+0000 7f109e66e740 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefe4e80 0x5616eeffd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.483+0000 7f109e66e740 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefda5e0 0x5616eeffcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.483+0000 7f109e66e740 1 --2- 172.21.15.46:0/2672087212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefd9cc0 0x5616eefe25d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.483+0000 7f109e66e740 1 -- 172.21.15.46:0/2672087212 >> 172.21.15.46:0/2672087212 conn(0x5616eefd6230 msgr2=0x5616eefdd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.489+0000 7f109e66e740 1 -- 172.21.15.46:0/2672087212 shutdown_connections 2024-09-12T03:02:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.491+0000 7f109e66e740 1 -- 172.21.15.46:0/2672087212 wait complete. 2024-09-12T03:02:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.491+0000 7f109e66e740 1 -- start start 2024-09-12T03:02:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.492+0000 7f109e66e740 1 Processor -- start 2024-09-12T03:02:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.492+0000 7f109e66e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefd9cc0 0x5616eefece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.492+0000 7f109e66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.493+0000 7f109e66e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefde070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.493+0000 7f109e66e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f107800e5e0 con 0x5616eefe4e80 2024-09-12T03:02:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.493+0000 7f109e66e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f107800fa50 con 0x5616eefd9cc0 2024-09-12T03:02:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.493+0000 7f109e66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f107800ecb0 con 0x5616eefda5e0 2024-09-12T03:02:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.493+0000 7f109e66e740 10 client.0 _ll_register_callbacks cb 0x7f10900096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:52144/0 (socket says 172.21.15.46:52144) 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 -- 172.21.15.46:0/432043092 learned_addr learned my addr 172.21.15.46:0/432043092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f109959b640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefd9cc0 0x5616eefece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 -- 172.21.15.46:0/432043092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefd9cc0 msgr2=0x5616eefece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefd9cc0 0x5616eefece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 -- 172.21.15.46:0/432043092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 msgr2=0x5616eefde070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefde070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.495+0000 7f1098d9a640 1 -- 172.21.15.46:0/432043092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616eefde5b0 con 0x5616eefda5e0 2024-09-12T03:02:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.498+0000 7f1098d9a640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f109002c010 tx=0x7f1090005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4950cc1d89f6a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.499+0000 7f107cff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:02:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.499+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090057060 con 0x5616eefda5e0 2024-09-12T03:02:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.499+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1084035070 con 0x5616eefda5e0 2024-09-12T03:02:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.500+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f1090004270 con 0x5616eefda5e0 2024-09-12T03:02:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.500+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f109005e050 con 0x5616eefda5e0 2024-09-12T03:02:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.501+0000 7f109e66e740 10 client.25789 fetch_fsmap learned FSMap version 35 2024-09-12T03:02:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.501+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5616ef05f5d0 con 0x5616eefda5e0 2024-09-12T03:02:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1084035070 con 0x5616eefda5e0 2024-09-12T03:02:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f10900035f0 con 0x5616eefda5e0 2024-09-12T03:02:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 10 client.25789 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T03:02:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 10 client.25789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 20 client.25789 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616ef05fd90 0x5616ef0621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 4 client.25789 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:02:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.505+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616eeec2900 con 0x5616ef05fd90 2024-09-12T03:02:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.518+0000 7f109959b640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616ef05fd90 0x5616ef0621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.520+0000 7f109959b640 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616ef05fd90 0x5616ef0621b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ee4ffb93d12a5f server_cookie=45b9f8a9d201eb9d in_seq=0 out_seq=0 2024-09-12T03:02:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.523+0000 7f107cff9640 10 client.25789 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.751+0000 7f107cff9640 1 -- 172.21.15.46:0/432043092 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616eeec2900 con 0x5616ef05fd90 2024-09-12T03:02:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.751+0000 7f107cff9640 10 client.25789 handle_command_reply: tid=0 2024-09-12T03:02:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.760+0000 7f109e66e740 1 client.25789 shutdown 2024-09-12T03:02:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.760+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 msgr2=0x5616eefed3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.760+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f109002c010 tx=0x7f1090005d30 comp rx=0 tx=0).stop 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 shutdown_connections 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5616ef05fd90 0x5616ef0621b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616eefe4e80 0x5616eefde070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616eefda5e0 0x5616eefed3d0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f109002c010 tx=0x7f1090005d30 comp rx=0 tx=0).stop 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 --2- 172.21.15.46:0/432043092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5616eefd9cc0 0x5616eefece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 >> 172.21.15.46:0/432043092 conn(0x5616eefd6230 msgr2=0x5616eefdb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f107cff9640 0 client.25789 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.765+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 shutdown_connections 2024-09-12T03:02:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.768+0000 7f109e66e740 1 -- 172.21.15.46:0/432043092 wait complete. 2024-09-12T03:02:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.768+0000 7f109e66e740 20 client.25789 trim_cache size 0 max 16384 2024-09-12T03:02:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:02:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:02:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:02:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:02:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.774+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 msgr2=0x7f10800a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.774+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 0x7f10800a5560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10840058c0 tx=0x7f108400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.774+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 msgr2=0x7f10941ab820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.774+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f108c009fd0 tx=0x7f108c00c800 comp rx=0 tx=0).stop 2024-09-12T03:02:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 shutdown_connections 2024-09-12T03:02:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10800a30a0 0x7f10800a5560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f109411b140 0x7f10941ab820 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941095d0 0x7f10941a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 --2- 172.21.15.46:0/4182916127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10941091f0 0x7f10941a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.782+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 >> 172.21.15.46:0/4182916127 conn(0x7f109406cbf0 msgr2=0x7f109411b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.783+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 shutdown_connections 2024-09-12T03:02:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:09.783+0000 7f109a59d640 1 -- 172.21.15.46:0/4182916127 wait complete. 2024-09-12T03:02:10.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:10 smithi046 ceph-mon[30951]: pgmap v375: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-12T03:02:10.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:10 smithi145 ceph-mon[34674]: pgmap v375: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-12T03:02:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:10 smithi196 ceph-mon[34587]: pgmap v375: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-12T03:02:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:12 smithi145 ceph-mon[34674]: pgmap v376: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2024-09-12T03:02:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:12 smithi196 ceph-mon[34587]: pgmap v376: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2024-09-12T03:02:12.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:12 smithi046 ceph-mon[30951]: pgmap v376: 129 pgs: 129 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2024-09-12T03:02:14.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:14 smithi145 ceph-mon[34674]: pgmap v377: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:14 smithi196 ceph-mon[34587]: pgmap v377: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:14.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:14 smithi046 ceph-mon[30951]: pgmap v377: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:16.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:16 smithi145 ceph-mon[34674]: pgmap v378: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:02:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:16 smithi196 ceph-mon[34587]: pgmap v378: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:02:16.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:16 smithi046 ceph-mon[30951]: pgmap v378: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:02:17.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:17 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:17.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:17 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:17.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:17 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:18.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:18 smithi145 ceph-mon[34674]: pgmap v379: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:02:18.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:18 smithi046 ceph-mon[30951]: pgmap v379: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:02:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:18 smithi196 ceph-mon[34587]: pgmap v379: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 4.16k op/s 2024-09-12T03:02:20.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:20 smithi145 ceph-mon[34674]: pgmap v380: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:02:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:20 smithi196 ceph-mon[34587]: pgmap v380: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:02:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:20 smithi046 ceph-mon[30951]: pgmap v380: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-12T03:02:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:22 smithi196 ceph-mon[34587]: pgmap v381: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-12T03:02:22.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:22 smithi046 ceph-mon[30951]: pgmap v381: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-12T03:02:22.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:22 smithi145 ceph-mon[34674]: pgmap v381: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-12T03:02:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.024+0000 7f09e67fc640 1 -- 172.21.15.46:0/1874283968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031640 con 0x7f09e81095d0 2024-09-12T03:02:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.026+0000 7f09edd86640 1 -- 172.21.15.46:0/1874283968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e81095d0 msgr2=0x7f09e8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.026+0000 7f09edd86640 1 --2- 172.21.15.46:0/1874283968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e81095d0 0x7f09e8109a10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009ca0 tx=0x7f09e002a8b0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.033+0000 7f09edd86640 1 -- 172.21.15.46:0/1874283968 shutdown_connections 2024-09-12T03:02:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.033+0000 7f09edd86640 1 --2- 172.21.15.46:0/1874283968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e811b140 0x7f09e811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.033+0000 7f09edd86640 1 --2- 172.21.15.46:0/1874283968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e81095d0 0x7f09e8109a10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.033+0000 7f09edd86640 1 --2- 172.21.15.46:0/1874283968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e81091f0 0x7f09e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.033+0000 7f09edd86640 1 -- 172.21.15.46:0/1874283968 >> 172.21.15.46:0/1874283968 conn(0x7f09e806cc60 msgr2=0x7f09e806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.040+0000 7f09edd86640 1 -- 172.21.15.46:0/1874283968 shutdown_connections 2024-09-12T03:02:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.042+0000 7f09edd86640 1 -- 172.21.15.46:0/1874283968 wait complete. 2024-09-12T03:02:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.043+0000 7f09edd86640 1 Processor -- start 2024-09-12T03:02:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 -- start start 2024-09-12T03:02:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e81095d0 0x7f09e81a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e811b140 0x7f09e81ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8120a10 con 0x7f09e811b140 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8120890 con 0x7f09e81091f0 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.044+0000 7f09edd86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8120b90 con 0x7f09e81095d0 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.046+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e811b140 0x7f09e81ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.046+0000 7f09e77fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.046+0000 7f09e77fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:36526/0 (socket says 172.21.15.46:36526) 2024-09-12T03:02:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.046+0000 7f09e77fe640 1 -- 172.21.15.46:0/555788918 learned_addr learned my addr 172.21.15.46:0/555788918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.052+0000 7f09e77fe640 1 -- 172.21.15.46:0/555788918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e81095d0 msgr2=0x7f09e81a7500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:02:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.052+0000 7f09e77fe640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e81095d0 0x7f09e81a7500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.052+0000 7f09e77fe640 1 -- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e811b140 msgr2=0x7f09e81ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.052+0000 7f09e77fe640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e811b140 0x7f09e81ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.052+0000 7f09e77fe640 1 -- 172.21.15.46:0/555788918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e81abf80 con 0x7f09e81091f0 2024-09-12T03:02:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.057+0000 7f09e77fe640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c8c0 tx=0x7f09dc00cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.060+0000 7f09e4ff9640 1 -- 172.21.15.46:0/555788918 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc004220 con 0x7f09e81091f0 2024-09-12T03:02:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.060+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e81ac270 con 0x7f09e81091f0 2024-09-12T03:02:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.060+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e8110920 con 0x7f09e81091f0 2024-09-12T03:02:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.061+0000 7f09e4ff9640 1 -- 172.21.15.46:0/555788918 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09dc0043c0 con 0x7f09e81091f0 2024-09-12T03:02:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.061+0000 7f09e4ff9640 1 -- 172.21.15.46:0/555788918 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc004a00 con 0x7f09e81091f0 2024-09-12T03:02:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.068+0000 7f09f0bfe740 1 Processor -- start 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 -- start start 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bc5e0 0x55a4244c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4244202f0 con 0x55a4244bbcc0 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a4243c8270 con 0x55a4244c6e80 2024-09-12T03:02:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.069+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a42441c2b0 con 0x55a4244bc5e0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.071+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.071+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51444/0 (socket says 172.21.15.46:51444) 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.071+0000 7f09e7fff640 1 -- 172.21.15.46:0/1345545546 learned_addr learned my addr 172.21.15.46:0/1345545546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e77fe640 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bc5e0 0x55a4244c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e7fff640 1 -- 172.21.15.46:0/1345545546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bc5e0 msgr2=0x55a4244c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e4ff9640 1 -- 172.21.15.46:0/555788918 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09dc004ba0 con 0x7f09e81091f0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bc5e0 0x55a4244c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e7fff640 1 -- 172.21.15.46:0/1345545546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 msgr2=0x55a4244c9270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c9270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.075+0000 7f09e7fff640 1 -- 172.21.15.46:0/1345545546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4243c7340 con 0x55a4244bbcc0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.076+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f09d800ca90 tx=0x7f09d800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da03fc218f2f764d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.076+0000 7f09e4ff9640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 0x7f09d00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.077+0000 7f09e4ff9640 1 -- 172.21.15.46:0/555788918 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f09dc0cf8c0 con 0x7f09e81091f0 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.077+0000 7f09e77fe640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 0x7f09d00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.078+0000 7f09e77fe640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 0x7f09d00a5380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a4244b9090 tx=0x7f09dc000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.078+0000 7f09cdffb640 1 -- 172.21.15.46:0/1345545546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004020 con 0x55a4244bbcc0 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.078+0000 7f09cdffb640 1 -- 172.21.15.46:0/1345545546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d80041c0 con 0x55a4244bbcc0 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.079+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1345545546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 msgr2=0x55a4244bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.079+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f09d800ca90 tx=0x7f09d800cf50 comp rx=0 tx=0).stop 2024-09-12T03:02:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.086+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1345545546 shutdown_connections 2024-09-12T03:02:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.086+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.086+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bc5e0 0x55a4244c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.086+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1345545546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244bc0a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.086+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1345545546 >> 172.21.15.46:0/1345545546 conn(0x55a4244b8230 msgr2=0x55a4244b8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.089+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1345545546 shutdown_connections 2024-09-12T03:02:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.089+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1345545546 wait complete. 2024-09-12T03:02:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.089+0000 7f09f0bfe740 1 -- start start 2024-09-12T03:02:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 Processor -- start 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4244c9aa0 con 0x55a4244bbcc0 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a4243a9290 con 0x55a4244bc5e0 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.090+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4244c9c20 con 0x55a4244c6e80 2024-09-12T03:02:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.091+0000 7f09f0bfe740 10 client.0 _ll_register_callbacks cb 0x55a4243da140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.091+0000 7f09e6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.091+0000 7f09e6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:36544/0 (socket says 172.21.15.46:36544) 2024-09-12T03:02:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.091+0000 7f09e6ffd640 1 -- 172.21.15.46:0/3225012264 learned_addr learned my addr 172.21.15.46:0/3225012264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.091+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 -- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 msgr2=0x55a4244c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 -- 172.21.15.46:0/3225012264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 msgr2=0x55a4244c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 -- 172.21.15.46:0/3225012264 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a424133210 con 0x55a4244bc5e0 2024-09-12T03:02:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09e6ffd640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09e0030da0 tx=0x7f09e0030dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21da423df8fd85f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.092+0000 7f09c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:02:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.093+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031640 con 0x55a4244bc5e0 2024-09-12T03:02:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.093+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d80041c0 con 0x55a4244bc5e0 2024-09-12T03:02:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.093+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f09e0030e70 con 0x55a4244bc5e0 2024-09-12T03:02:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.093+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f09e004d050 con 0x55a4244bc5e0 2024-09-12T03:02:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.096+0000 7f09f0bfe740 10 client.25823 fetch_fsmap learned FSMap version 36 2024-09-12T03:02:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.096+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a4241ff620 con 0x55a4244bc5e0 2024-09-12T03:02:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.096+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d80041c0 con 0x55a4244bc5e0 2024-09-12T03:02:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.096+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f09e0044470 con 0x55a4244bc5e0 2024-09-12T03:02:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 10 client.25823 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 20 client.25823 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a4244d6ea0 0x55a4244d92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 4 client.25823 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42442da20 con 0x55a4244d6ea0 2024-09-12T03:02:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.097+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a4244d6ea0 0x55a4244d92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.100+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a4244d6ea0 0x55a4244d92e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679530f74933946 server_cookie=d4ff0be394d2453b in_seq=0 out_seq=0 2024-09-12T03:02:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.102+0000 7f09c6ffd640 10 client.25823 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.104+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3225012264 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a42442da20 con 0x55a4244d6ea0 2024-09-12T03:02:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.104+0000 7f09c6ffd640 10 client.25823 handle_command_reply: tid=0 2024-09-12T03:02:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.104+0000 7f09f0bfe740 1 client.25823 shutdown 2024-09-12T03:02:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.106+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 msgr2=0x55a4244c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.106+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09e0030da0 tx=0x7f09e0030dd0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 shutdown_connections 2024-09-12T03:02:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a4244d6ea0 0x55a4244d92e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244bc5e0 0x55a4244c4220 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f09e0030da0 tx=0x7f09e0030dd0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3225012264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 >> 172.21.15.46:0/3225012264 conn(0x55a4244b8230 msgr2=0x55a4244bdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.108+0000 7f09c6ffd640 0 client.25823 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.116+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 shutdown_connections 2024-09-12T03:02:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.116+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3225012264 wait complete. 2024-09-12T03:02:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.118+0000 7f09f0bfe740 20 client.25823 trim_cache size 0 max 16384 2024-09-12T03:02:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.166+0000 7f09f0bfe740 1 Processor -- start 2024-09-12T03:02:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 -- start start 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 0x55a4244debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09d40128a0 con 0x55a4244bbcc0 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4010110 con 0x55a4244be730 2024-09-12T03:02:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.168+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09d400df80 con 0x55a4244c6e80 2024-09-12T03:02:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.173+0000 7f09e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.173+0000 7f09e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:41450/0 (socket says 172.21.15.46:41450) 2024-09-12T03:02:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.173+0000 7f09e6ffd640 1 -- 172.21.15.46:0/1097353448 learned_addr learned my addr 172.21.15.46:0/1097353448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.175+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.175+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 0x55a4244debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.177+0000 7f09e7fff640 1 -- 172.21.15.46:0/1097353448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 msgr2=0x55a4244b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.177+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.177+0000 7f09e7fff640 1 -- 172.21.15.46:0/1097353448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 msgr2=0x55a4244c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.177+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.177+0000 7f09e7fff640 1 -- 172.21.15.46:0/1097353448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4244b2450 con 0x55a4244be730 2024-09-12T03:02:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.178+0000 7f09e7fff640 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 0x55a4244debf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a4244d9b90 tx=0x7f09d800cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34f676536e03d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.178+0000 7f09cd7fa640 1 -- 172.21.15.46:0/1097353448 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800ee60 con 0x55a4244be730 2024-09-12T03:02:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.179+0000 7f09cd7fa640 1 -- 172.21.15.46:0/1097353448 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d80035f0 con 0x55a4244be730 2024-09-12T03:02:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.180+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1097353448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 msgr2=0x55a4244debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.180+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 0x55a4244debf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a4244d9b90 tx=0x7f09d800cf50 comp rx=0 tx=0).stop 2024-09-12T03:02:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.184+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1097353448 shutdown_connections 2024-09-12T03:02:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.184+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244be730 0x55a4244debf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.184+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244c6e80 0x55a4244b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.184+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/1097353448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244bbcc0 0x55a4244c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.184+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1097353448 >> 172.21.15.46:0/1097353448 conn(0x55a4244b8230 msgr2=0x55a4244b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.187+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1097353448 shutdown_connections 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.191+0000 7f09f0bfe740 1 -- 172.21.15.46:0/1097353448 wait complete. 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.191+0000 7f09f0bfe740 1 -- start start 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.191+0000 7f09f0bfe740 1 Processor -- start 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244be730 0x55a4244c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09d400e5d0 con 0x55a4244be730 2024-09-12T03:02:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09d400fa40 con 0x55a4244c6e80 2024-09-12T03:02:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09d400eca0 con 0x55a4244bbcc0 2024-09-12T03:02:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.193+0000 7f09f0bfe740 10 client.0 _ll_register_callbacks cb 0x7f09e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.195+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.195+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.195+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:41458/0 (socket says 172.21.15.46:41458) 2024-09-12T03:02:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.195+0000 7f09e7fff640 1 -- 172.21.15.46:0/3897685944 learned_addr learned my addr 172.21.15.46:0/3897685944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 -- 172.21.15.46:0/3897685944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 msgr2=0x55a4244c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 -- 172.21.15.46:0/3897685944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244be730 msgr2=0x55a4244c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244be730 0x55a4244c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 -- 172.21.15.46:0/3897685944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4244cd930 con 0x55a4244bbcc0 2024-09-12T03:02:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.196+0000 7f09e7fff640 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005940 tx=0x7f09d800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2266ee9f1ddcfd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.198+0000 7f09c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:02:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.198+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8035070 con 0x55a4244bbcc0 2024-09-12T03:02:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.198+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9423+0+0 (secure 0 0 0) 0x7f09d80037a0 con 0x55a4244bbcc0 2024-09-12T03:02:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.201+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d80035f0 con 0x55a4244bbcc0 2024-09-12T03:02:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.203+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f09d80039a0 con 0x55a4244bbcc0 2024-09-12T03:02:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.203+0000 7f09f0bfe740 10 client.25795 fetch_fsmap learned FSMap version 36 2024-09-12T03:02:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.203+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a4245420b0 con 0x55a4244bbcc0 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d80035f0 con 0x55a4244bbcc0 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f09d8029910 con 0x55a4244bbcc0 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 10 client.25795 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 20 client.25795 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a424542740 0x55a424544b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 4 client.25795 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.209+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a424427d70 con 0x55a424542740 2024-09-12T03:02:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.210+0000 7f09e6ffd640 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a424542740 0x55a424544b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.211+0000 7f09e6ffd640 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a424542740 0x55a424544b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d0f5c7302f3df9 server_cookie=bc55bb414f9d73cf in_seq=0 out_seq=0 2024-09-12T03:02:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:23.211+0000 7f09c6ffd640 10 client.25795 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09c6ffd640 1 -- 172.21.15.46:0/3897685944 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a424427d70 con 0x55a424542740 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09c6ffd640 10 client.25795 handle_command_reply: tid=0 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 client.25795 shutdown 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 msgr2=0x55a4244c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005940 tx=0x7f09d800c9c0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 shutdown_connections 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a424542740 0x55a424544b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4244c6e80 0x55a4244c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4244be730 0x55a4244c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.047+0000 7f09f0bfe740 1 --2- 172.21.15.46:0/3897685944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4244bbcc0 0x55a4244c81e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.048+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 >> 172.21.15.46:0/3897685944 conn(0x55a4244b8230 msgr2=0x55a4244bdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.048+0000 7f09c6ffd640 0 client.25795 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.048+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 shutdown_connections 2024-09-12T03:02:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.048+0000 7f09f0bfe740 1 -- 172.21.15.46:0/3897685944 wait complete. 2024-09-12T03:02:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.048+0000 7f09f0bfe740 20 client.25795 trim_cache size 0 max 16384 2024-09-12T03:02:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:02:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:02:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:02:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 msgr2=0x7f09d00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 0x7f09d00a5380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a4244b9090 tx=0x7f09dc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 msgr2=0x7f09e81a6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c8c0 tx=0x7f09dc00cd80 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 shutdown_connections 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f09d00a2ec0 0x7f09d00a5380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e811b140 0x7f09e81ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e81095d0 0x7f09e81a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 --2- 172.21.15.46:0/555788918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09e81091f0 0x7f09e81a6fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 >> 172.21.15.46:0/555788918 conn(0x7f09e806cc60 msgr2=0x7f09e811b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 shutdown_connections 2024-09-12T03:02:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:24.051+0000 7f09edd86640 1 -- 172.21.15.46:0/555788918 wait complete. 2024-09-12T03:02:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:24 smithi196 ceph-mon[34587]: pgmap v382: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:02:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:24 smithi046 ceph-mon[30951]: pgmap v382: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:02:24.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:24 smithi145 ceph-mon[34674]: pgmap v382: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2024-09-12T03:02:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:26 smithi196 ceph-mon[34587]: pgmap v383: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.43k op/s 2024-09-12T03:02:26.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:26 smithi046 ceph-mon[30951]: pgmap v383: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.43k op/s 2024-09-12T03:02:26.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:26 smithi145 ceph-mon[34674]: pgmap v383: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 3.43k op/s 2024-09-12T03:02:27.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:27 smithi196 ceph-mon[34587]: pgmap v384: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.18k op/s 2024-09-12T03:02:27.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:27 smithi046 ceph-mon[30951]: pgmap v384: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.18k op/s 2024-09-12T03:02:27.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:27 smithi145 ceph-mon[34674]: pgmap v384: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.18k op/s 2024-09-12T03:02:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:28 smithi196 ceph-mon[34587]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:02:28.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:28 smithi145 ceph-mon[34674]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:02:28.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:28 smithi046 ceph-mon[30951]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:02:29.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:29 smithi196 ceph-mon[34587]: pgmap v386: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:29.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:29 smithi046 ceph-mon[30951]: pgmap v386: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:29.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:29 smithi145 ceph-mon[34674]: pgmap v386: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:32.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:31 smithi046 ceph-mon[30951]: pgmap v387: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:32.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:31 smithi145 ceph-mon[34674]: pgmap v387: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:32.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:31 smithi196 ceph-mon[34587]: pgmap v387: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-12T03:02:34.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:33 smithi046 ceph-mon[30951]: pgmap v388: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:33 smithi145 ceph-mon[34674]: pgmap v388: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:33 smithi196 ceph-mon[34587]: pgmap v388: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:36.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:35 smithi046 ceph-mon[30951]: pgmap v389: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:36.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:35 smithi145 ceph-mon[34674]: pgmap v389: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:36.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:35 smithi196 ceph-mon[34587]: pgmap v389: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:02:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.219+0000 7f10e344c640 1 -- 172.21.15.46:0/2194369066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc1095d0 msgr2=0x7f10dc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.219+0000 7f10e344c640 1 --2- 172.21.15.46:0/2194369066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc1095d0 0x7f10dc109a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009890 tx=0x7f10d802a390 comp rx=0 tx=0).stop 2024-09-12T03:02:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 -- 172.21.15.46:0/2194369066 shutdown_connections 2024-09-12T03:02:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 --2- 172.21.15.46:0/2194369066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc11b140 0x7f10dc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 --2- 172.21.15.46:0/2194369066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc1095d0 0x7f10dc109a10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 --2- 172.21.15.46:0/2194369066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 -- 172.21.15.46:0/2194369066 >> 172.21.15.46:0/2194369066 conn(0x7f10dc06ce00 msgr2=0x7f10dc06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 -- 172.21.15.46:0/2194369066 shutdown_connections 2024-09-12T03:02:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.222+0000 7f10e344c640 1 -- 172.21.15.46:0/2194369066 wait complete. 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.223+0000 7f10e344c640 1 Processor -- start 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.223+0000 7f10e344c640 1 -- start start 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc1095d0 0x7f10dc1109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc11b140 0x7f10dc110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc120be0 con 0x7f10dc1091f0 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc120a60 con 0x7f10dc11b140 2024-09-12T03:02:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e344c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc120d60 con 0x7f10dc1095d0 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49954/0 (socket says 172.21.15.46:49954) 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e11e8640 1 -- 172.21.15.46:0/3331326148 learned_addr learned my addr 172.21.15.46:0/3331326148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.224+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc11b140 0x7f10dc110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e11e8640 1 -- 172.21.15.46:0/3331326148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc1095d0 msgr2=0x7f10dc1109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e09e7640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc1095d0 0x7f10dc1109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc1095d0 0x7f10dc1109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e11e8640 1 -- 172.21.15.46:0/3331326148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc11b140 msgr2=0x7f10dc110f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc11b140 0x7f10dc110f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.225+0000 7f10e11e8640 1 -- 172.21.15.46:0/3331326148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc111750 con 0x7f10dc1091f0 2024-09-12T03:02:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.231+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b4a0 tx=0x7f10cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.235+0000 7f10d27fc640 1 -- 172.21.15.46:0/3331326148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc014070 con 0x7f10dc1091f0 2024-09-12T03:02:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.235+0000 7f10d27fc640 1 -- 172.21.15.46:0/3331326148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10cc010ac0 con 0x7f10dc1091f0 2024-09-12T03:02:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.235+0000 7f10d27fc640 1 -- 172.21.15.46:0/3331326148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc017c30 con 0x7f10dc1091f0 2024-09-12T03:02:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.235+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1bbef0 con 0x7f10dc1091f0 2024-09-12T03:02:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.235+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1bc360 con 0x7f10dc1091f0 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.239+0000 7f10e61e3740 1 Processor -- start 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 -- start start 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcbff7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc0050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560dcbf5c2f0 con 0x560dcbff7af0 2024-09-12T03:02:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560dcbf04270 con 0x560dcbff8410 2024-09-12T03:02:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.241+0000 7f10e61e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560dcbf582b0 con 0x560dcc002cb0 2024-09-12T03:02:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.243+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.243+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56934/0 (socket says 172.21.15.46:56934) 2024-09-12T03:02:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.243+0000 7f10e11e8640 1 -- 172.21.15.46:0/1618215251 learned_addr learned my addr 172.21.15.46:0/1618215251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10d27fc640 1 -- 172.21.15.46:0/3331326148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f10cc017070 con 0x7f10dc1091f0 2024-09-12T03:02:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10e11e8640 1 -- 172.21.15.46:0/1618215251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 msgr2=0x560dcc0050a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:02:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10e11e8640 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc0050a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10e11e8640 1 -- 172.21.15.46:0/1618215251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 msgr2=0x560dcbff7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10e11e8640 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcbff7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.245+0000 7f10e11e8640 1 -- 172.21.15.46:0/1618215251 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dcbf03340 con 0x560dcbff8410 2024-09-12T03:02:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.246+0000 7f10d27fc640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 0x7f10c80a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.247+0000 7f10d27fc640 1 -- 172.21.15.46:0/3331326148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f10cc0cf140 con 0x7f10dc1091f0 2024-09-12T03:02:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.247+0000 7f10e11e8640 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0cc470 tx=0x7f10cc0d2e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0b5cbf09c19f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.248+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 0x7f10c80a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.248+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 0x7f10c80a5490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009ba0 tx=0x7f10d4010040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.249+0000 7f10c77fe640 1 -- 172.21.15.46:0/1618215251 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01e070 con 0x560dcbff8410 2024-09-12T03:02:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.249+0000 7f10c77fe640 1 -- 172.21.15.46:0/1618215251 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10cc09ba00 con 0x560dcbff8410 2024-09-12T03:02:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.249+0000 7f10c77fe640 1 -- 172.21.15.46:0/1618215251 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc09ae40 con 0x560dcbff8410 2024-09-12T03:02:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.250+0000 7f10e61e3740 1 -- 172.21.15.46:0/1618215251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 msgr2=0x560dcc002770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.250+0000 7f10e61e3740 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0cc470 tx=0x7f10cc0d2e50 comp rx=0 tx=0).stop 2024-09-12T03:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.253+0000 7f10e61e3740 1 -- 172.21.15.46:0/1618215251 shutdown_connections 2024-09-12T03:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.253+0000 7f10e61e3740 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc0050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.253+0000 7f10e61e3740 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc002770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.253+0000 7f10e61e3740 1 --2- 172.21.15.46:0/1618215251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcbff7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.253+0000 7f10e61e3740 1 -- 172.21.15.46:0/1618215251 >> 172.21.15.46:0/1618215251 conn(0x560dcbff4060 msgr2=0x560dcbff4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.256+0000 7f10e61e3740 1 -- 172.21.15.46:0/1618215251 shutdown_connections 2024-09-12T03:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.258+0000 7f10e61e3740 1 -- 172.21.15.46:0/1618215251 wait complete. 2024-09-12T03:02:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.259+0000 7f10e61e3740 1 -- start start 2024-09-12T03:02:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.259+0000 7f10e61e3740 1 Processor -- start 2024-09-12T03:02:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff7af0 0x560dcbfffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff8410 0x560dcc000130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560dcc005880 con 0x560dcc002cb0 2024-09-12T03:02:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560dcbee5290 con 0x560dcbff7af0 2024-09-12T03:02:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560dcc005a00 con 0x560dcbff8410 2024-09-12T03:02:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.261+0000 7f10e61e3740 10 client.0 _ll_register_callbacks cb 0x560dcbf16140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.262+0000 7f10e19e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.262+0000 7f10e19e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49966/0 (socket says 172.21.15.46:49966) 2024-09-12T03:02:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.262+0000 7f10e19e9640 1 -- 172.21.15.46:0/3343027191 learned_addr learned my addr 172.21.15.46:0/3343027191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.262+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff7af0 0x560dcbfffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e09e7640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff8410 0x560dcc000130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 -- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff8410 msgr2=0x560dcc000130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff8410 0x560dcc000130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 -- 172.21.15.46:0/3343027191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff7af0 msgr2=0x560dcbfffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff7af0 0x560dcbfffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 -- 172.21.15.46:0/3343027191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dcbc6f210 con 0x560dcc002cb0 2024-09-12T03:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10e19e9640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009b00 tx=0x7f10d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f7468b389534fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.263+0000 7f10c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.264+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d401a9e0 con 0x560dcc002cb0 2024-09-12T03:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.264+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10cc09ba00 con 0x560dcc002cb0 2024-09-12T03:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.264+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f10d4020030 con 0x560dcc002cb0 2024-09-12T03:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.265+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f10d4030050 con 0x560dcc002cb0 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.265+0000 7f10e61e3740 10 client.16257 fetch_fsmap learned FSMap version 36 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.265+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560dcbd3b620 con 0x560dcc002cb0 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.266+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10cc09ba00 con 0x560dcc002cb0 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f10d400b040 con 0x560dcc002cb0 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 10 client.16257 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 20 client.16257 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc013250 0x560dcc015690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 4 client.16257 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.267+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dcbf69a20 con 0x560dcc013250 2024-09-12T03:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.268+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc013250 0x560dcc015690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.272+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc013250 0x560dcc015690 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1be5043044f418 server_cookie=df0bfa151a72bfff in_seq=0 out_seq=0 2024-09-12T03:02:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.272+0000 7f10c4ff9640 10 client.16257 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.273+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3343027191 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dcbf69a20 con 0x560dcc013250 2024-09-12T03:02:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.273+0000 7f10c4ff9640 10 client.16257 handle_command_reply: tid=0 2024-09-12T03:02:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.273+0000 7f10e61e3740 1 client.16257 shutdown 2024-09-12T03:02:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.273+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 msgr2=0x560dcc000670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.273+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009b00 tx=0x7f10d4004ab0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 shutdown_connections 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc013250 0x560dcc015690 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcc002cb0 0x560dcc000670 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff8410 0x560dcc000130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3343027191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff7af0 0x560dcbfffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 >> 172.21.15.46:0/3343027191 conn(0x560dcbff4060 msgr2=0x560dcbff9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10c4ff9640 0 client.16257 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 shutdown_connections 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 1 -- 172.21.15.46:0/3343027191 wait complete. 2024-09-12T03:02:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.274+0000 7f10e61e3740 20 client.16257 trim_cache size 0 max 16384 2024-09-12T03:02:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.305+0000 7f10e61e3740 1 Processor -- start 2024-09-12T03:02:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.305+0000 7f10e61e3740 1 -- start start 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.305+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc01afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e11e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49970/0 (socket says 172.21.15.46:49970) 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e11e8640 1 -- 172.21.15.46:0/3035389795 learned_addr learned my addr 172.21.15.46:0/3035389795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc01b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e09e7640 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc01afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.306+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10c00128b0 con 0x560dcbff7af0 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0010120 con 0x560dcbff8410 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10c000e020 con 0x560dcc002cb0 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e11e8640 1 -- 172.21.15.46:0/3035389795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 msgr2=0x560dcc01b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc01b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e11e8640 1 -- 172.21.15.46:0/3035389795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 msgr2=0x560dcc01afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc01afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.307+0000 7f10e11e8640 1 -- 172.21.15.46:0/3035389795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dcbff7870 con 0x560dcbff7af0 2024-09-12T03:02:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.308+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0d74b0 tx=0x7f10cc0d2e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1777f7c12c75fc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.310+0000 7f10c6ffd640 1 -- 172.21.15.46:0/3035389795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0b3020 con 0x560dcbff7af0 2024-09-12T03:02:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.310+0000 7f10c6ffd640 1 -- 172.21.15.46:0/3035389795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10cc0107b0 con 0x560dcbff7af0 2024-09-12T03:02:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.311+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 msgr2=0x560dcc0004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.311+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0d74b0 tx=0x7f10cc0d2e50 comp rx=0 tx=0).stop 2024-09-12T03:02:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.313+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 shutdown_connections 2024-09-12T03:02:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.313+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcc002cb0 0x560dcc01b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.313+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcbff8410 0x560dcc01afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.313+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3035389795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff7af0 0x560dcc0004a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.313+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 >> 172.21.15.46:0/3035389795 conn(0x560dcbff4060 msgr2=0x560dcc001c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.316+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 shutdown_connections 2024-09-12T03:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.317+0000 7f10e61e3740 1 -- 172.21.15.46:0/3035389795 wait complete. 2024-09-12T03:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.318+0000 7f10e61e3740 1 -- start start 2024-09-12T03:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.318+0000 7f10e61e3740 1 Processor -- start 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 0x560dcc00b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff8410 0x560dcbffc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 0x560dcbffcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10c000e5e0 con 0x560dcbff8410 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10c000fa50 con 0x560dcc002cb0 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10c000ecb0 con 0x560dcbff7af0 2024-09-12T03:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e61e3740 10 client.0 _ll_register_callbacks cb 0x7f10d4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e19e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 0x560dcbffcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e19e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 0x560dcbffcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56958/0 (socket says 172.21.15.46:56958) 2024-09-12T03:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e19e9640 1 -- 172.21.15.46:0/3756493519 learned_addr learned my addr 172.21.15.46:0/3756493519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.319+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 0x560dcc00b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 -- 172.21.15.46:0/3756493519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 msgr2=0x560dcbffcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 0x560dcbffcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 -- 172.21.15.46:0/3756493519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff8410 msgr2=0x560dcbffc740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff8410 0x560dcbffc740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 -- 172.21.15.46:0/3756493519 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dcbffd2c0 con 0x560dcbff7af0 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10e11e8640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 0x560dcc00b1f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0d74b0 tx=0x7f10cc09b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca644378cf858a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.320+0000 7f10c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.324+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0b3070 con 0x560dcbff7af0 2024-09-12T03:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.324+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f10cc0d2850 con 0x560dcbff7af0 2024-09-12T03:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.325+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560dcbf63d70 con 0x560dcbff7af0 2024-09-12T03:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.326+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f10cc09ac60 con 0x560dcbff7af0 2024-09-12T03:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.328+0000 7f10e61e3740 10 client.25804 fetch_fsmap learned FSMap version 36 2024-09-12T03:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.328+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560dcc07ddb0 con 0x560dcbff7af0 2024-09-12T03:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.329+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560dcbf63d70 con 0x560dcbff7af0 2024-09-12T03:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f10cc097970 con 0x560dcbff7af0 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 10 client.25804 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 20 client.25804 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc07e2b0 0x560dcc0806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 4 client.25804 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dcbee0900 con 0x560dcc07e2b0 2024-09-12T03:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.330+0000 7f10e09e7640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc07e2b0 0x560dcc0806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.332+0000 7f10c4ff9640 10 client.25804 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.332+0000 7f10e09e7640 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc07e2b0 0x560dcc0806d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578ac725d2076361 server_cookie=3eab5a5bccc5f62d in_seq=0 out_seq=0 2024-09-12T03:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.492+0000 7f10c4ff9640 1 -- 172.21.15.46:0/3756493519 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dcbee0900 con 0x560dcc07e2b0 2024-09-12T03:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.492+0000 7f10c4ff9640 10 client.25804 handle_command_reply: tid=0 2024-09-12T03:02:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.496+0000 7f10e61e3740 1 client.25804 shutdown 2024-09-12T03:02:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.497+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 msgr2=0x560dcc00b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.497+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 0x560dcc00b1f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0d74b0 tx=0x7f10cc09b160 comp rx=0 tx=0).stop 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 shutdown_connections 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x560dcc07e2b0 0x560dcc0806d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560dcc002cb0 0x560dcbffcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dcbff8410 0x560dcbffc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 --2- 172.21.15.46:0/3756493519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560dcbff7af0 0x560dcc00b1f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 >> 172.21.15.46:0/3756493519 conn(0x560dcbff4060 msgr2=0x560dcbff29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10c4ff9640 0 client.25804 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 shutdown_connections 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 1 -- 172.21.15.46:0/3756493519 wait complete. 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.498+0000 7f10e61e3740 20 client.25804 trim_cache size 0 max 16384 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:02:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 msgr2=0x7f10c80a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 0x7f10c80a5490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009ba0 tx=0x7f10d4010040 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 msgr2=0x7f10dc118a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b4a0 tx=0x7f10cc00b960 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 shutdown_connections 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f10c80a2fd0 0x7f10c80a5490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10dc11b140 0x7f10dc110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10dc1095d0 0x7f10dc1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 --2- 172.21.15.46:0/3331326148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10dc1091f0 0x7f10dc118a30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.503+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 >> 172.21.15.46:0/3331326148 conn(0x7f10dc06ce00 msgr2=0x7f10dc11b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.504+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 shutdown_connections 2024-09-12T03:02:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:37.504+0000 7f10e344c640 1 -- 172.21.15.46:0/3331326148 wait complete. 2024-09-12T03:02:38.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:37 smithi046 ceph-mon[30951]: pgmap v390: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2024-09-12T03:02:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:37 smithi145 ceph-mon[34674]: pgmap v390: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2024-09-12T03:02:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:37 smithi196 ceph-mon[34587]: pgmap v390: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2024-09-12T03:02:40.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:39 smithi046 ceph-mon[30951]: pgmap v391: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:39 smithi145 ceph-mon[34674]: pgmap v391: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:39 smithi196 ceph-mon[34587]: pgmap v391: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2024-09-12T03:02:42.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:41 smithi046 ceph-mon[30951]: pgmap v392: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-12T03:02:42.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:41 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:41 smithi145 ceph-mon[34674]: pgmap v392: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-12T03:02:42.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:41 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:41 smithi196 ceph-mon[34587]: pgmap v392: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 3.12k op/s 2024-09-12T03:02:42.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:41 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:43.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:42 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:43.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:42 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:43.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:42 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:44.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:43 smithi145 ceph-mon[34674]: pgmap v393: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:02:44.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:43 smithi196 ceph-mon[34587]: pgmap v393: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:02:44.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:43 smithi046 ceph-mon[30951]: pgmap v393: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2024-09-12T03:02:46.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:45 smithi145 ceph-mon[34674]: pgmap v394: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:02:46.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:45 smithi196 ceph-mon[34587]: pgmap v394: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:02:46.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:45 smithi046 ceph-mon[30951]: pgmap v394: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:02:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:47 smithi145 ceph-mon[34674]: pgmap v395: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:02:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:48.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:02:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:47 smithi196 ceph-mon[34587]: pgmap v395: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:02:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:02:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:47 smithi046 ceph-mon[30951]: pgmap v395: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:02:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:48.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:02:50.088 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:49 smithi046 ceph-mon[30951]: pgmap v396: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-12T03:02:50.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:49 smithi145 ceph-mon[34674]: pgmap v396: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-12T03:02:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:49 smithi196 ceph-mon[34587]: pgmap v396: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-12T03:02:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.689+0000 7f8849f82640 1 -- 172.21.15.46:0/745229090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 msgr2=0x7f883c0a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.689+0000 7f8849f82640 1 --2- 172.21.15.46:0/745229090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c0a1420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8844067960 tx=0x7f8844096c00 comp rx=0 tx=0).stop 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 -- 172.21.15.46:0/745229090 shutdown_connections 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 --2- 172.21.15.46:0/745229090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c0a1420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 --2- 172.21.15.46:0/745229090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c095520 0x7f883c099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 --2- 172.21.15.46:0/745229090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c094b70 0x7f883c094f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 -- 172.21.15.46:0/745229090 >> 172.21.15.46:0/745229090 conn(0x7f883c00a1e0 msgr2=0x7f883c00a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.690+0000 7f8849f82640 1 -- 172.21.15.46:0/745229090 shutdown_connections 2024-09-12T03:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.691+0000 7f8849f82640 1 -- 172.21.15.46:0/745229090 wait complete. 2024-09-12T03:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.692+0000 7f8849f82640 1 Processor -- start 2024-09-12T03:02:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.693+0000 7f8849f82640 1 -- start start 2024-09-12T03:02:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.693+0000 7f8849f82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 0x7f883c12f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.693+0000 7f8849f82640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8848f80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 0x7f883c12f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8843fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8843fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:33790/0 (socket says 172.21.15.46:33790) 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8843fff640 1 -- 172.21.15.46:0/3279156558 learned_addr learned my addr 172.21.15.46:0/3279156558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c129920 0x7f883c129dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f883c0a35d0 con 0x7f883c129920 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f883c0a3450 con 0x7f883c099880 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.694+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f883c0a3750 con 0x7f883c094b70 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 -- 172.21.15.46:0/3279156558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 msgr2=0x7f883c12f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 0x7f883c12f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 -- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c129920 msgr2=0x7f883c129dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c129920 0x7f883c129dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c12a440 con 0x7f883c099880 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8848f80640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 0x7f883c12f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8843fff640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f883000c970 tx=0x7f883000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8841ffb640 1 -- 172.21.15.46:0/3279156558 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830007bb0 con 0x7f883c099880 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8841ffb640 1 -- 172.21.15.46:0/3279156558 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8830007d50 con 0x7f883c099880 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8841ffb640 1 -- 172.21.15.46:0/3279156558 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000f450 con 0x7f883c099880 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.695+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c12a730 con 0x7f883c099880 2024-09-12T03:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.696+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c136930 con 0x7f883c099880 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.701+0000 7f884e056740 1 Processor -- start 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.701+0000 7f884e056740 1 -- start start 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae4fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 0x55571ae65910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55571adb42f0 con 0x55571ae63520 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55571ad5c270 con 0x55571ae50080 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.702+0000 7f884e056740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55571adb02b0 con 0x55571ae51240 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.703+0000 7f8849781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae4fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.703+0000 7f8841ffb640 1 -- 172.21.15.46:0/3279156558 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f88300046c0 con 0x7f883c099880 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.703+0000 7f8848f80640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.703+0000 7f8848f80640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:33802/0 (socket says 172.21.15.46:33802) 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.703+0000 7f8848f80640 1 -- 172.21.15.46:0/4273205626 learned_addr learned my addr 172.21.15.46:0/4273205626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.704+0000 7f8843fff640 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 0x55571ae65910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.704+0000 7f8848f80640 1 -- 172.21.15.46:0/4273205626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 msgr2=0x55571ae4fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.704+0000 7f8848f80640 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae4fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.704+0000 7f8848f80640 1 -- 172.21.15.46:0/4273205626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 msgr2=0x55571ae65910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.704+0000 7f8841ffb640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 0x7f88340a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8849781640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 0x7f88340a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8848f80640 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 0x55571ae65910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8848f80640 1 -- 172.21.15.46:0/4273205626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571ad5b340 con 0x55571ae50080 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8843fff640 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 0x55571ae65910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8841ffb640 1 -- 172.21.15.46:0/3279156558 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f88300cedb0 con 0x7f883c099880 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8849781640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 0x7f88340a5300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8844000b60 tx=0x7f8844079040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.705+0000 7f8848f80640 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f883800b770 tx=0x7f8838005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6998979d9bdf3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.706+0000 7f882effd640 1 -- 172.21.15.46:0/4273205626 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838019020 con 0x55571ae50080 2024-09-12T03:02:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.706+0000 7f882effd640 1 -- 172.21.15.46:0/4273205626 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015ab0 con 0x55571ae50080 2024-09-12T03:02:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.706+0000 7f882effd640 1 -- 172.21.15.46:0/4273205626 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838014a00 con 0x55571ae50080 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.706+0000 7f884e056740 1 -- 172.21.15.46:0/4273205626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 msgr2=0x55571ae504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.706+0000 7f884e056740 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f883800b770 tx=0x7f8838005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 -- 172.21.15.46:0/4273205626 shutdown_connections 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae63520 0x55571ae65910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae504e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 --2- 172.21.15.46:0/4273205626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae4fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 -- 172.21.15.46:0/4273205626 >> 172.21.15.46:0/4273205626 conn(0x55571ae4c230 msgr2=0x55571ae4c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 -- 172.21.15.46:0/4273205626 shutdown_connections 2024-09-12T03:02:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 -- 172.21.15.46:0/4273205626 wait complete. 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.707+0000 7f884e056740 1 -- start start 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.708+0000 7f884e056740 1 Processor -- start 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.708+0000 7f884e056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.708+0000 7f884e056740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f884e056740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae63520 0x55571ae54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f8843fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f8843fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:41658/0 (socket says 172.21.15.46:41658) 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f8843fff640 1 -- 172.21.15.46:0/1658060945 learned_addr learned my addr 172.21.15.46:0/1658060945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f8849781640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae63520 0x55571ae54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55571ae66300 con 0x55571ae50080 2024-09-12T03:02:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55571ad3d290 con 0x55571ae63520 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f8848f80640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.709+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55571ae66480 con 0x55571ae51240 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f884e056740 10 client.0 _ll_register_callbacks cb 0x55571ad6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 -- 172.21.15.46:0/1658060945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae63520 msgr2=0x55571ae54270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae63520 0x55571ae54270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 -- 172.21.15.46:0/1658060945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 msgr2=0x55571ae537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571aac7210 con 0x55571ae51240 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f8843fff640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88300058c0 tx=0x7f883009aa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d836eb6c9a81303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.710+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838015ab0 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f8823fff640 10 client.25813 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f8823fff640 1 -- 172.21.15.46:0/1658060945 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883009b560 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f8823fff640 1 -- 172.21.15.46:0/1658060945 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f88300974d0 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f8823fff640 1 -- 172.21.15.46:0/1658060945 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f883009f210 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f884e056740 10 client.25813 fetch_fsmap learned FSMap version 37 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.711+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55571ab93620 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f8823fff640 1 -- 172.21.15.46:0/1658060945 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f88300a4b50 con 0x55571ae51240 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 10 client.25813 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 20 client.25813 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571ae739b0 0x55571ae75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 4 client.25813 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.712+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8838015ab0 con 0x55571ae739b0 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.713+0000 7f8848f80640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571ae739b0 0x55571ae75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.713+0000 7f8848f80640 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571ae739b0 0x55571ae75e10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff813cc8fd9bdd5 server_cookie=b1ccbda7aa3e4c7c in_seq=0 out_seq=0 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.713+0000 7f8823fff640 10 client.25813 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.715+0000 7f8823fff640 1 -- 172.21.15.46:0/1658060945 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8838015ab0 con 0x55571ae739b0 2024-09-12T03:02:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.715+0000 7f8823fff640 10 client.25813 handle_command_reply: tid=0 2024-09-12T03:02:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.715+0000 7f884e056740 1 client.25813 shutdown 2024-09-12T03:02:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.715+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 msgr2=0x55571ae53d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.715+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88300058c0 tx=0x7f883009aa60 comp rx=0 tx=0).stop 2024-09-12T03:02:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 shutdown_connections 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571ae739b0 0x55571ae75e10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae63520 0x55571ae54270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae51240 0x55571ae53d30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 --2- 172.21.15.46:0/1658060945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 >> 172.21.15.46:0/1658060945 conn(0x55571ae4c230 msgr2=0x55571ae51f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.717+0000 7f8823fff640 0 client.25813 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.720+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 shutdown_connections 2024-09-12T03:02:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.720+0000 7f884e056740 1 -- 172.21.15.46:0/1658060945 wait complete. 2024-09-12T03:02:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.721+0000 7f884e056740 20 client.25813 trim_cache size 0 max 16384 2024-09-12T03:02:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.749+0000 7f884e056740 1 Processor -- start 2024-09-12T03:02:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.749+0000 7f884e056740 1 -- start start 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.749+0000 7f884e056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.749+0000 7f884e056740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 0x55571ae52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.749+0000 7f884e056740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f884e056740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1432e0 con 0x55571ae50080 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f884e056740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f883c140b80 con 0x55571ae51240 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f884e056740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f883c13e9f0 con 0x55571ae63520 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8849781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8849781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:41672/0 (socket says 172.21.15.46:41672) 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40806/0 (socket says 172.21.15.46:40806) 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8849781640 1 -- 172.21.15.46:0/545606286 learned_addr learned my addr 172.21.15.46:0/545606286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8843fff640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 0x55571ae52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 -- 172.21.15.46:0/545606286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 msgr2=0x55571ae6e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 -- 172.21.15.46:0/545606286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 msgr2=0x55571ae52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 0x55571ae52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8848f80640 1 -- 172.21.15.46:0/545606286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571ae56a40 con 0x55571ae50080 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.750+0000 7f8849781640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.751+0000 7f8843fff640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 0x55571ae52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.751+0000 7f8848f80640 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f883800bd50 tx=0x7f8838014360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1114948977f59f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.751+0000 7f882e7fc640 1 -- 172.21.15.46:0/545606286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838019040 con 0x55571ae50080 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.751+0000 7f882e7fc640 1 -- 172.21.15.46:0/545606286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015ab0 con 0x55571ae50080 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.751+0000 7f882e7fc640 1 -- 172.21.15.46:0/545606286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380148d0 con 0x55571ae50080 2024-09-12T03:02:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 -- 172.21.15.46:0/545606286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 msgr2=0x55571ae46ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f883800bd50 tx=0x7f8838014360 comp rx=0 tx=0).stop 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 -- 172.21.15.46:0/545606286 shutdown_connections 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae51240 0x55571ae52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 --2- 172.21.15.46:0/545606286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae50080 0x55571ae46ed0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.752+0000 7f884e056740 1 -- 172.21.15.46:0/545606286 >> 172.21.15.46:0/545606286 conn(0x55571ae4c230 msgr2=0x55571ae4c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.755+0000 7f884e056740 1 -- 172.21.15.46:0/545606286 shutdown_connections 2024-09-12T03:02:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.755+0000 7f884e056740 1 -- 172.21.15.46:0/545606286 wait complete. 2024-09-12T03:02:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.755+0000 7f884e056740 1 -- start start 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.755+0000 7f884e056740 1 Processor -- start 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.756+0000 7f884e056740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.756+0000 7f884e056740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.756+0000 7f884e056740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.756+0000 7f8843fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f8843fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40808/0 (socket says 172.21.15.46:40808) 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f8843fff640 1 -- 172.21.15.46:0/4275711914 learned_addr learned my addr 172.21.15.46:0/4275711914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f883c13f040 con 0x55571ae51240 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f8849781640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1404b0 con 0x55571ae50080 2024-09-12T03:02:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.758+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f883c13f710 con 0x55571ae63520 2024-09-12T03:02:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.758+0000 7f884e056740 10 client.0 _ll_register_callbacks cb 0x7f883009ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.757+0000 7f8848f80640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 -- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 msgr2=0x55571ae6bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 -- 172.21.15.46:0/4275711914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 msgr2=0x55571ae61db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae61db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571ae6c2f0 con 0x55571ae51240 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8843fff640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f883009adf0 tx=0x7f88300d1240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd5824f4d66b7af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8823fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883009b560 con 0x55571ae51240 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f8830099d20 con 0x55571ae51240 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.759+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838015ab0 con 0x55571ae51240 2024-09-12T03:02:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.760+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8830082050 con 0x55571ae51240 2024-09-12T03:02:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.761+0000 7f884e056740 10 client.16272 fetch_fsmap learned FSMap version 37 2024-09-12T03:02:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.761+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55571aede6b0 con 0x55571ae51240 2024-09-12T03:02:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.762+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8838015ab0 con 0x55571ae51240 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f8830099f20 con 0x55571ae51240 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 10 client.16272 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 20 client.16272 populate_metadata read hostname 'smithi046' 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571aededb0 0x55571aee11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 4 client.16272 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:02:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55571ae653d0 con 0x55571aededb0 2024-09-12T03:02:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.763+0000 7f8848f80640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571aededb0 0x55571aee11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.766+0000 7f8848f80640 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571aededb0 0x55571aee11d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc92de044a59e0 server_cookie=b1ed316b4edd0f46 in_seq=0 out_seq=0 2024-09-12T03:02:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:50.766+0000 7f8823fff640 10 client.16272 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.065+0000 7f8823fff640 1 -- 172.21.15.46:0/4275711914 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55571ae653d0 con 0x55571aededb0 2024-09-12T03:02:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.065+0000 7f8823fff640 10 client.16272 handle_command_reply: tid=0 2024-09-12T03:02:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.066+0000 7f884e056740 1 client.16272 shutdown 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 msgr2=0x55571ae6b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f883009adf0 tx=0x7f88300d1240 comp rx=0 tx=0).stop 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 shutdown_connections 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55571aededb0 0x55571aee11d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55571ae63520 0x55571ae6bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55571ae51240 0x55571ae6b770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 --2- 172.21.15.46:0/4275711914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55571ae50080 0x55571ae61db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 >> 172.21.15.46:0/4275711914 conn(0x55571ae4c230 msgr2=0x55571ae6e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.075+0000 7f8823fff640 0 client.16272 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:02:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.077+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 shutdown_connections 2024-09-12T03:02:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.077+0000 7f884e056740 1 -- 172.21.15.46:0/4275711914 wait complete. 2024-09-12T03:02:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.077+0000 7f884e056740 20 client.16272 trim_cache size 0 max 16384 2024-09-12T03:02:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:02:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:02:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:02:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:02:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.081+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 msgr2=0x7f88340a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.081+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 0x7f88340a5300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8844000b60 tx=0x7f8844079040 comp rx=0 tx=0).stop 2024-09-12T03:02:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.081+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 msgr2=0x7f883c12fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.082+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f883000c970 tx=0x7f883000ce30 comp rx=0 tx=0).stop 2024-09-12T03:02:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 shutdown_connections 2024-09-12T03:02:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f88340a2e40 0x7f88340a5300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883c129920 0x7f883c129dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f883c099880 0x7f883c12fcb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 --2- 172.21.15.46:0/3279156558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f883c094b70 0x7f883c12f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.084+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 >> 172.21.15.46:0/3279156558 conn(0x7f883c00a1e0 msgr2=0x7f883c092bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.085+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 shutdown_connections 2024-09-12T03:02:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:02:51.086+0000 7f8849f82640 1 -- 172.21.15.46:0/3279156558 wait complete. 2024-09-12T03:02:52.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:51 smithi145 ceph-mon[34674]: pgmap v397: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:02:52.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:51 smithi196 ceph-mon[34587]: pgmap v397: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:02:52.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:51 smithi046 ceph-mon[30951]: pgmap v397: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:02:54.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:53 smithi145 ceph-mon[34674]: pgmap v398: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2024-09-12T03:02:54.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:53 smithi196 ceph-mon[34587]: pgmap v398: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2024-09-12T03:02:54.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:53 smithi046 ceph-mon[30951]: pgmap v398: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2024-09-12T03:02:56.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:55 smithi145 ceph-mon[34674]: pgmap v399: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:02:56.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:55 smithi196 ceph-mon[34587]: pgmap v399: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:02:56.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:55 smithi046 ceph-mon[30951]: pgmap v399: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:02:58.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:57 smithi145 ceph-mon[34674]: pgmap v400: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.68k op/s 2024-09-12T03:02:58.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:57 smithi196 ceph-mon[34587]: pgmap v400: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.68k op/s 2024-09-12T03:02:58.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:57 smithi046 ceph-mon[30951]: pgmap v400: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 3.68k op/s 2024-09-12T03:03:00.109 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:02:59 smithi145 ceph-mon[34674]: pgmap v401: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:00.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:02:59 smithi196 ceph-mon[34587]: pgmap v401: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:00.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:02:59 smithi046 ceph-mon[30951]: pgmap v401: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:02.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:01 smithi145 ceph-mon[34674]: pgmap v402: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-12T03:03:02.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:01 smithi196 ceph-mon[34587]: pgmap v402: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-12T03:03:02.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:01 smithi046 ceph-mon[30951]: pgmap v402: 129 pgs: 4 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-09-12T03:03:03.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:02 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:03.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:02 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:03.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:02 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:04.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:03 smithi196 ceph-mon[34587]: pgmap v403: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:03:04.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:03 smithi046 ceph-mon[30951]: pgmap v403: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:03:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.303+0000 7f122bb31640 1 -- 172.21.15.46:0/912975989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241095d0 msgr2=0x7f1224109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.303+0000 7f122bb31640 1 --2- 172.21.15.46:0/912975989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241095d0 0x7f1224109a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f121c009930 tx=0x7f121c02a440 comp rx=0 tx=0).stop 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.310+0000 7f122bb31640 1 -- 172.21.15.46:0/912975989 shutdown_connections 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.310+0000 7f122bb31640 1 --2- 172.21.15.46:0/912975989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122411b140 0x7f122411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.310+0000 7f122bb31640 1 --2- 172.21.15.46:0/912975989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241095d0 0x7f1224109a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.310+0000 7f122bb31640 1 --2- 172.21.15.46:0/912975989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12241091f0 0x7f1224110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.310+0000 7f122bb31640 1 -- 172.21.15.46:0/912975989 >> 172.21.15.46:0/912975989 conn(0x7f122406ce00 msgr2=0x7f122406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.311+0000 7f122bb31640 1 -- 172.21.15.46:0/912975989 shutdown_connections 2024-09-12T03:03:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.311+0000 7f122bb31640 1 -- 172.21.15.46:0/912975989 wait complete. 2024-09-12T03:03:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.312+0000 7f122bb31640 1 Processor -- start 2024-09-12T03:03:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.312+0000 7f122bb31640 1 -- start start 2024-09-12T03:03:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.313+0000 7f122bb31640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 0x7f1224118570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.313+0000 7f122bb31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12241095d0 0x7f1224118ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.313+0000 7f122bb31640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 0x7f1224111690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.313+0000 7f12298cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 0x7f1224118570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.313+0000 7f12298cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 0x7f1224118570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37788/0 (socket says 172.21.15.46:37788) 2024-09-12T03:03:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.314+0000 7f12298cd640 1 -- 172.21.15.46:0/3894383143 learned_addr learned my addr 172.21.15.46:0/3894383143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.314+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 0x7f1224111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.314+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1224120ba0 con 0x7f122411b140 2024-09-12T03:03:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.314+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1224120a20 con 0x7f12241091f0 2024-09-12T03:03:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.314+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1224120d20 con 0x7f12241095d0 2024-09-12T03:03:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.315+0000 7f12290cc640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12241095d0 0x7f1224118ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 -- 172.21.15.46:0/3894383143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12241095d0 msgr2=0x7f1224118ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12241095d0 0x7f1224118ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 -- 172.21.15.46:0/3894383143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 msgr2=0x7f1224118570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 0x7f1224118570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1224111f50 con 0x7f122411b140 2024-09-12T03:03:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.316+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 0x7f1224111690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f122000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.319+0000 7f1216ffd640 1 -- 172.21.15.46:0/3894383143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122000ce20 con 0x7f122411b140 2024-09-12T03:03:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.319+0000 7f1216ffd640 1 -- 172.21.15.46:0/3894383143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1220005020 con 0x7f122411b140 2024-09-12T03:03:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.319+0000 7f1216ffd640 1 -- 172.21.15.46:0/3894383143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220010670 con 0x7f122411b140 2024-09-12T03:03:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.319+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1224112240 con 0x7f122411b140 2024-09-12T03:03:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.320+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1224116270 con 0x7f122411b140 2024-09-12T03:03:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.326+0000 7f122e8d2740 1 Processor -- start 2024-09-12T03:03:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f1216ffd640 1 -- 172.21.15.46:0/3894383143 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12200051d0 con 0x7f122411b140 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 -- start start 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 0x564afa1b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564afa1132f0 con 0x564afa1aeaf0 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564afa0bb270 con 0x564afa1af410 2024-09-12T03:03:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.327+0000 7f122e8d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564afa10f2b0 con 0x564afa1b9cb0 2024-09-12T03:03:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.328+0000 7f12298cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 0x564afa1b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.328+0000 7f12298cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 0x564afa1b9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37804/0 (socket says 172.21.15.46:37804) 2024-09-12T03:03:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.328+0000 7f12298cd640 1 -- 172.21.15.46:0/1428692971 learned_addr learned my addr 172.21.15.46:0/1428692971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.328+0000 7f12290cc640 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.329+0000 7f12290cc640 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.331+0000 7f1216ffd640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 0x7f12100a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f12290cc640 1 -- 172.21.15.46:0/1428692971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 msgr2=0x564afa1b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f12290cc640 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 0x564afa1b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f12290cc640 1 -- 172.21.15.46:0/1428692971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 msgr2=0x564afa1aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f12290cc640 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f12290cc640 1 -- 172.21.15.46:0/1428692971 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afa0ba340 con 0x564afa1b9cb0 2024-09-12T03:03:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.332+0000 7f1216ffd640 1 -- 172.21.15.46:0/3894383143 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f12200cfc80 con 0x7f122411b140 2024-09-12T03:03:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.333+0000 7f12290cc640 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1bc0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3bb00480568ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.333+0000 7f120ffff640 1 -- 172.21.15.46:0/1428692971 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c035070 con 0x564afa1b9cb0 2024-09-12T03:03:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.333+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 0x7f12100a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.333+0000 7f120ffff640 1 -- 172.21.15.46:0/1428692971 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f121c031de0 con 0x564afa1b9cb0 2024-09-12T03:03:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.334+0000 7f122e8d2740 1 -- 172.21.15.46:0/1428692971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 msgr2=0x564afa1bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.334+0000 7f122e8d2740 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1bc0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 -- 172.21.15.46:0/1428692971 shutdown_connections 2024-09-12T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1bc0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1af410 0x564afa1b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 --2- 172.21.15.46:0/1428692971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 -- 172.21.15.46:0/1428692971 >> 172.21.15.46:0/1428692971 conn(0x564afa1ab060 msgr2=0x564afa1ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 -- 172.21.15.46:0/1428692971 shutdown_connections 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122a0ce640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 0x7f12100a5480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1220009730 tx=0x7f12200096c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.339+0000 7f122e8d2740 1 -- 172.21.15.46:0/1428692971 wait complete. 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.340+0000 7f122e8d2740 1 -- start start 2024-09-12T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.340+0000 7f122e8d2740 1 Processor -- start 2024-09-12T03:03:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.341+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.341+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.341+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.341+0000 7f122e8d2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564afa1bc8d0 con 0x564afa1af410 2024-09-12T03:03:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f122e8d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564afa09c290 con 0x564afa1aeaf0 2024-09-12T03:03:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f122e8d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564afa1bca50 con 0x564afa1b9cb0 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f122e8d2740 10 client.0 _ll_register_callbacks cb 0x564afa0cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f12290cc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f12290cc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37810/0 (socket says 172.21.15.46:37810) 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f12290cc640 1 -- 172.21.15.46:0/2392685579 learned_addr learned my addr 172.21.15.46:0/2392685579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.342+0000 7f12290cc640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.343+0000 7f12298cd640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.344+0000 7f12290cc640 1 -- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 msgr2=0x564afa1b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.344+0000 7f12290cc640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.344+0000 7f12290cc640 1 -- 172.21.15.46:0/2392685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 msgr2=0x564afa1b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.344+0000 7f12290cc640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.344+0000 7f12290cc640 1 -- 172.21.15.46:0/2392685579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af9e26210 con 0x564afa1aeaf0 2024-09-12T03:03:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.348+0000 7f12290cc640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f121c004070 tx=0x7f121c02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f655a338874ae6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.351+0000 7f120d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.351+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c050020 con 0x564afa1aeaf0 2024-09-12T03:03:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.351+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f121c004b40 con 0x564afa1aeaf0 2024-09-12T03:03:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.351+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f121c031de0 con 0x564afa1aeaf0 2024-09-12T03:03:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.355+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f121c035070 con 0x564afa1aeaf0 2024-09-12T03:03:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.355+0000 7f122e8d2740 10 client.25874 fetch_fsmap learned FSMap version 38 2024-09-12T03:03:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.355+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564af9ef2620 con 0x564afa1aeaf0 2024-09-12T03:03:04.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:03 smithi145 ceph-mon[34674]: pgmap v403: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2024-09-12T03:03:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.359+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f121c031de0 con 0x564afa1aeaf0 2024-09-12T03:03:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.360+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f121c03a040 con 0x564afa1aeaf0 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 10 client.25874 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 20 client.25874 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa1c9b20 0x564afa1cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 4 client.25874 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.361+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564afa120a20 con 0x564afa1c9b20 2024-09-12T03:03:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.362+0000 7f12298cd640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa1c9b20 0x564afa1cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.366+0000 7f12298cd640 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa1c9b20 0x564afa1cbf10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94962d594ca04c54 server_cookie=9c6b87f43da9d62b in_seq=0 out_seq=0 2024-09-12T03:03:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.367+0000 7f120d7fa640 10 client.25874 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.368+0000 7f120d7fa640 1 -- 172.21.15.46:0/2392685579 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564afa120a20 con 0x564afa1c9b20 2024-09-12T03:03:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.368+0000 7f120d7fa640 10 client.25874 handle_command_reply: tid=0 2024-09-12T03:03:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.369+0000 7f122e8d2740 1 client.25874 shutdown 2024-09-12T03:03:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.376+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 msgr2=0x564afa1b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.376+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f121c004070 tx=0x7f121c02a440 comp rx=0 tx=0).stop 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 shutdown_connections 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa1c9b20 0x564afa1cbf10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 --2- 172.21.15.46:0/2392685579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b6b80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 >> 172.21.15.46:0/2392685579 conn(0x564afa1ab060 msgr2=0x564afa1b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f120d7fa640 0 client.25874 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 shutdown_connections 2024-09-12T03:03:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.379+0000 7f122e8d2740 1 -- 172.21.15.46:0/2392685579 wait complete. 2024-09-12T03:03:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.380+0000 7f122e8d2740 20 client.25874 trim_cache size 0 max 16384 2024-09-12T03:03:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.409+0000 7f122e8d2740 1 Processor -- start 2024-09-12T03:03:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 -- start start 2024-09-12T03:03:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c8e50 con 0x564afa1af410 2024-09-12T03:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c66f0 con 0x564afa1aeaf0 2024-09-12T03:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.410+0000 7f122e8d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c45f0 con 0x564afa1b9cb0 2024-09-12T03:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12298cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:37818/0 (socket says 172.21.15.46:37818) 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 -- 172.21.15.46:0/543807539 learned_addr learned my addr 172.21.15.46:0/543807539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 -- 172.21.15.46:0/543807539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 msgr2=0x564afa1d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 -- 172.21.15.46:0/543807539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 msgr2=0x564afa1b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.411+0000 7f12290cc640 1 -- 172.21.15.46:0/543807539 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afa1bc760 con 0x564afa1aeaf0 2024-09-12T03:03:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.412+0000 7f12290cc640 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f121c03d480 tx=0x7f121c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e091a66a72461120 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.413+0000 7f120f7fe640 1 -- 172.21.15.46:0/543807539 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c050030 con 0x564afa1aeaf0 2024-09-12T03:03:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.413+0000 7f120f7fe640 1 -- 172.21.15.46:0/543807539 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f121c051040 con 0x564afa1aeaf0 2024-09-12T03:03:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.413+0000 7f120f7fe640 1 -- 172.21.15.46:0/543807539 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c003750 con 0x564afa1aeaf0 2024-09-12T03:03:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.414+0000 7f122e8d2740 1 -- 172.21.15.46:0/543807539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 msgr2=0x564afa1b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.414+0000 7f122e8d2740 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f121c03d480 tx=0x7f121c031400 comp rx=0 tx=0).stop 2024-09-12T03:03:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.417+0000 7f122e8d2740 1 -- 172.21.15.46:0/543807539 shutdown_connections 2024-09-12T03:03:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.417+0000 7f122e8d2740 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1b9cb0 0x564afa1d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.417+0000 7f122e8d2740 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1af410 0x564afa1b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.417+0000 7f122e8d2740 1 --2- 172.21.15.46:0/543807539 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1aeaf0 0x564afa1b15b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.417+0000 7f122e8d2740 1 -- 172.21.15.46:0/543807539 >> 172.21.15.46:0/543807539 conn(0x564afa1ab060 msgr2=0x564afa1b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.422+0000 7f122e8d2740 1 -- 172.21.15.46:0/543807539 shutdown_connections 2024-09-12T03:03:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.423+0000 7f122e8d2740 1 -- 172.21.15.46:0/543807539 wait complete. 2024-09-12T03:03:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 -- start start 2024-09-12T03:03:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 Processor -- start 2024-09-12T03:03:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1af410 0x564afa1c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 0x564afa1b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c4bb0 con 0x564afa1aeaf0 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c6020 con 0x564afa1b9cb0 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.424+0000 7f122e8d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12241c5280 con 0x564afa1af410 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f122e8d2740 10 client.0 _ll_register_callbacks cb 0x7f121c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f12290cc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 0x564afa1b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f12290cc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f12290cc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1c1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53566/0 (socket says 172.21.15.46:53566) 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f12290cc640 1 -- 172.21.15.46:0/3116787190 learned_addr learned my addr 172.21.15.46:0/3116787190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.425+0000 7f12298cd640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1af410 0x564afa1c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 -- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1af410 msgr2=0x564afa1c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1af410 0x564afa1c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 -- 172.21.15.46:0/3116787190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 msgr2=0x564afa1c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 -- 172.21.15.46:0/3116787190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afa1b3710 con 0x564afa1b9cb0 2024-09-12T03:03:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f12290cc640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 0x564afa1b31a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f121c003de0 tx=0x7f121c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d100aeafc3bc52b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.426+0000 7f120d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c050040 con 0x564afa1b9cb0 2024-09-12T03:03:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f121c051040 con 0x564afa1b9cb0 2024-09-12T03:03:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9555+0+0 (secure 0 0 0) 0x7f121c055070 con 0x564afa1b9cb0 2024-09-12T03:03:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f121c075050 con 0x564afa1b9cb0 2024-09-12T03:03:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f122e8d2740 10 client.25880 fetch_fsmap learned FSMap version 38 2024-09-12T03:03:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.427+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564afa233d30 con 0x564afa1b9cb0 2024-09-12T03:03:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.428+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f121c051040 con 0x564afa1b9cb0 2024-09-12T03:03:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.428+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f121c064040 con 0x564afa1b9cb0 2024-09-12T03:03:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 10 client.25880 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:03:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 20 client.25880 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa2341f0 0x564afa236610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 4 client.25880 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:03:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.429+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564afa11ad70 con 0x564afa2341f0 2024-09-12T03:03:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.430+0000 7f12298cd640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa2341f0 0x564afa236610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.430+0000 7f12298cd640 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa2341f0 0x564afa236610 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9013df83d121877d server_cookie=bef9fc23204bddd9 in_seq=0 out_seq=0 2024-09-12T03:03:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.430+0000 7f120d7fa640 10 client.25880 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f120d7fa640 1 -- 172.21.15.46:0/3116787190 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564afa11ad70 con 0x564afa2341f0 2024-09-12T03:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f120d7fa640 10 client.25880 handle_command_reply: tid=0 2024-09-12T03:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 client.25880 shutdown 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 msgr2=0x564afa1b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 0x564afa1b31a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f121c003de0 tx=0x7f121c0035f0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 shutdown_connections 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x564afa2341f0 0x564afa236610 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564afa1b9cb0 0x564afa1b31a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564afa1af410 0x564afa1c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 --2- 172.21.15.46:0/3116787190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564afa1aeaf0 0x564afa1c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 >> 172.21.15.46:0/3116787190 conn(0x564afa1ab060 msgr2=0x564afa1a99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f120d7fa640 0 client.25880 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 shutdown_connections 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.987+0000 7f122e8d2740 1 -- 172.21.15.46:0/3116787190 wait complete. 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.988+0000 7f122e8d2740 20 client.25880 trim_cache size 0 max 16384 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 msgr2=0x7f12100a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 0x7f12100a5480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1220009730 tx=0x7f12200096c0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 msgr2=0x7f1224111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 0x7f1224111690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f122000ef10 comp rx=0 tx=0).stop 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 shutdown_connections 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f12100a2fc0 0x7f12100a5480 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122411b140 0x7f1224111690 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12241095d0 0x7f1224118ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 --2- 172.21.15.46:0/3894383143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12241091f0 0x7f1224118570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 >> 172.21.15.46:0/3894383143 conn(0x7f122406ce00 msgr2=0x7f122411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 shutdown_connections 2024-09-12T03:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:04.991+0000 7f122bb31640 1 -- 172.21.15.46:0/3894383143 wait complete. 2024-09-12T03:03:06.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:05 smithi196 ceph-mon[34587]: pgmap v404: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:06.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:05 smithi046 ceph-mon[30951]: pgmap v404: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:06.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:05 smithi145 ceph-mon[34674]: pgmap v404: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:07 smithi196 ceph-mon[34587]: pgmap v405: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-12T03:03:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:07 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:08.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:07 smithi046 ceph-mon[30951]: pgmap v405: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-12T03:03:08.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:07 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:08.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:07 smithi145 ceph-mon[34674]: pgmap v405: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-12T03:03:08.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:07 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:10.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:09 smithi196 ceph-mon[34587]: pgmap v406: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-12T03:03:10.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:09 smithi046 ceph-mon[30951]: pgmap v406: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-12T03:03:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:09 smithi145 ceph-mon[34674]: pgmap v406: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-12T03:03:11.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:11 smithi145 ceph-mon[34674]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:03:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:11 smithi196 ceph-mon[34587]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:03:11.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:11 smithi046 ceph-mon[30951]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:03:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:12 smithi145 ceph-mon[34674]: pgmap v408: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-12T03:03:12.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:12 smithi145 ceph-mon[34674]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:03:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:12 smithi196 ceph-mon[34587]: pgmap v408: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-12T03:03:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:12 smithi196 ceph-mon[34587]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:03:12.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:12 smithi046 ceph-mon[30951]: pgmap v408: 129 pgs: 129 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-12T03:03:12.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:12 smithi046 ceph-mon[30951]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:03:14.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:14 smithi145 ceph-mon[34674]: pgmap v410: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:03:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:14 smithi196 ceph-mon[34587]: pgmap v410: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:03:14.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:14 smithi046 ceph-mon[30951]: pgmap v410: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:03:16.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:16 smithi145 ceph-mon[34674]: pgmap v411: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:16 smithi196 ceph-mon[34587]: pgmap v411: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:16.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:16 smithi046 ceph-mon[30951]: pgmap v411: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.190+0000 7ff09933d640 1 -- 172.21.15.46:0/2131270282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09411b140 msgr2=0x7ff09411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.190+0000 7ff09933d640 1 --2- 172.21.15.46:0/2131270282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09411b140 0x7ff09411d530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff08c01acb0 tx=0x7ff08c03b900 comp rx=0 tx=0).stop 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.191+0000 7ff09933d640 1 -- 172.21.15.46:0/2131270282 shutdown_connections 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.191+0000 7ff09933d640 1 --2- 172.21.15.46:0/2131270282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff09411b140 0x7ff09411d530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.191+0000 7ff09933d640 1 --2- 172.21.15.46:0/2131270282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941095d0 0x7ff094109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.191+0000 7ff09933d640 1 --2- 172.21.15.46:0/2131270282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0941091f0 0x7ff094110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.191+0000 7ff09933d640 1 -- 172.21.15.46:0/2131270282 >> 172.21.15.46:0/2131270282 conn(0x7ff09406cbf0 msgr2=0x7ff09406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.196+0000 7ff09933d640 1 -- 172.21.15.46:0/2131270282 shutdown_connections 2024-09-12T03:03:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.204+0000 7ff09933d640 1 -- 172.21.15.46:0/2131270282 wait complete. 2024-09-12T03:03:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.204+0000 7ff09933d640 1 Processor -- start 2024-09-12T03:03:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 -- start start 2024-09-12T03:03:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0941095d0 0x7ff094116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff09411b140 0x7ff094116d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941131c0 con 0x7ff0941095d0 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff094113040 con 0x7ff0941091f0 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff09933d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff094113340 con 0x7ff09411b140 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff093fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff093fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56600/0 (socket says 172.21.15.46:56600) 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff093fff640 1 -- 172.21.15.46:0/1677677561 learned_addr learned my addr 172.21.15.46:0/1677677561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.205+0000 7ff098b3c640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff09411b140 0x7ff094116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 -- 172.21.15.46:0/1677677561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff09411b140 msgr2=0x7ff094116d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff09411b140 0x7ff094116d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 -- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0941095d0 msgr2=0x7ff094116840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:03:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0941095d0 0x7ff094116840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 -- 172.21.15.46:0/1677677561 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094117370 con 0x7ff0941091f0 2024-09-12T03:03:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.206+0000 7ff093fff640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ea20 tx=0x7ff08800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.207+0000 7ff0917fa640 1 -- 172.21.15.46:0/1677677561 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08800cd80 con 0x7ff0941091f0 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.207+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff094117660 con 0x7ff0941091f0 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.207+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941cd1d0 con 0x7ff0941091f0 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 Processor -- start 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 -- start start 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff945e80 0x5597ff948270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5597ff89f2f0 con 0x5597ff93acc0 2024-09-12T03:03:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5597ff847270 con 0x5597ff93b5e0 2024-09-12T03:03:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.211+0000 7ff09d418740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597ff89b2b0 con 0x5597ff945e80 2024-09-12T03:03:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.212+0000 7ff093fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.212+0000 7ff093fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56616/0 (socket says 172.21.15.46:56616) 2024-09-12T03:03:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.212+0000 7ff093fff640 1 -- 172.21.15.46:0/1505202020 learned_addr learned my addr 172.21.15.46:0/1505202020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff0917fa640 1 -- 172.21.15.46:0/1677677561 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0880108e0 con 0x7ff0941091f0 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff093fff640 1 -- 172.21.15.46:0/1505202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff945e80 msgr2=0x5597ff948270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff093fff640 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff945e80 0x5597ff948270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff093fff640 1 -- 172.21.15.46:0/1505202020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 msgr2=0x5597ff93b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff093fff640 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff093fff640 1 -- 172.21.15.46:0/1505202020 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ff846340 con 0x5597ff93b5e0 2024-09-12T03:03:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.213+0000 7ff0917fa640 1 -- 172.21.15.46:0/1677677561 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880177e0 con 0x7ff0941091f0 2024-09-12T03:03:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.216+0000 7ff0917fa640 1 -- 172.21.15.46:0/1677677561 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff088022a90 con 0x7ff0941091f0 2024-09-12T03:03:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.217+0000 7ff0917fa640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 0x7ff07c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.217+0000 7ff0917fa640 1 -- 172.21.15.46:0/1677677561 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7ff0880d1e20 con 0x7ff0941091f0 2024-09-12T03:03:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.218+0000 7ff093fff640 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff088017f40 tx=0x7ff088017f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432a1226f1743e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.219+0000 7ff07a7fc640 1 -- 172.21.15.46:0/1505202020 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08809a390 con 0x5597ff93b5e0 2024-09-12T03:03:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.219+0000 7ff07a7fc640 1 -- 172.21.15.46:0/1505202020 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff088010a90 con 0x5597ff93b5e0 2024-09-12T03:03:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.220+0000 7ff09d418740 1 -- 172.21.15.46:0/1505202020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 msgr2=0x5597ff945940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.220+0000 7ff09d418740 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff088017f40 tx=0x7ff088017f70 comp rx=0 tx=0).stop 2024-09-12T03:03:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.225+0000 7ff098b3c640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 0x7ff07c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.230+0000 7ff09d418740 1 -- 172.21.15.46:0/1505202020 shutdown_connections 2024-09-12T03:03:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.230+0000 7ff09d418740 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff945e80 0x5597ff948270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.230+0000 7ff09d418740 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff93b5e0 0x5597ff945940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.230+0000 7ff09d418740 1 --2- 172.21.15.46:0/1505202020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.230+0000 7ff09d418740 1 -- 172.21.15.46:0/1505202020 >> 172.21.15.46:0/1505202020 conn(0x5597ff937230 msgr2=0x5597ff937610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.240+0000 7ff09d418740 1 -- 172.21.15.46:0/1505202020 shutdown_connections 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.240+0000 7ff098b3c640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 0x7ff07c0a53c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005e30 tx=0x7ff08c005ef0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.245+0000 7ff09d418740 1 -- 172.21.15.46:0/1505202020 wait complete. 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.245+0000 7ff09d418740 1 -- start start 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.245+0000 7ff09d418740 1 Processor -- start 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93acc0 0x5597ff942da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93b5e0 0x5597ff9432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5597ff96aa60 con 0x5597ff93b5e0 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5597ff828290 con 0x5597ff945e80 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597ff96abe0 con 0x5597ff93acc0 2024-09-12T03:03:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff09d418740 10 client.0 _ll_register_callbacks cb 0x5597ff859140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff098b3c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff098b3c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56626/0 (socket says 172.21.15.46:56626) 2024-09-12T03:03:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.246+0000 7ff098b3c640 1 -- 172.21.15.46:0/3186598076 learned_addr learned my addr 172.21.15.46:0/3186598076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff093fff640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93acc0 0x5597ff942da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 -- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93acc0 msgr2=0x5597ff942da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93acc0 0x5597ff942da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 -- 172.21.15.46:0/3186598076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93b5e0 msgr2=0x5597ff9432e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93b5e0 0x5597ff9432e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 -- 172.21.15.46:0/3186598076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ff5b2210 con 0x5597ff945e80 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.247+0000 7ff098b3c640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff08c046000 tx=0x7ff08c003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1054894ffab3f387 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.248+0000 7ff0737fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.259+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c04e070 con 0x5597ff945e80 2024-09-12T03:03:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.259+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7ff08c041800 con 0x5597ff945e80 2024-09-12T03:03:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.261+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5597ff8aca20 con 0x5597ff945e80 2024-09-12T03:03:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.261+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff08c058050 con 0x5597ff945e80 2024-09-12T03:03:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.265+0000 7ff09d418740 10 client.25892 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.265+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597ff67e620 con 0x5597ff945e80 2024-09-12T03:03:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.266+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5597ff8aca20 con 0x5597ff945e80 2024-09-12T03:03:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.266+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ff08c0045d0 con 0x5597ff945e80 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.268+0000 7ff09d418740 10 client.25892 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.268+0000 7ff09d418740 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.268+0000 7ff09d418740 20 client.25892 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.269+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff978010 0x5597ff97a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.269+0000 7ff09d418740 4 client.25892 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.269+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ff8a6d70 con 0x5597ff978010 2024-09-12T03:03:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.274+0000 7ff093fff640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff978010 0x5597ff97a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.279+0000 7ff093fff640 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff978010 0x5597ff97a480 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109008cafe22be52 server_cookie=37db4fe2df46fd36 in_seq=0 out_seq=0 2024-09-12T03:03:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.286+0000 7ff0737fe640 10 client.25892 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.286+0000 7ff0737fe640 1 -- 172.21.15.46:0/3186598076 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597ff8a6d70 con 0x5597ff978010 2024-09-12T03:03:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.286+0000 7ff0737fe640 10 client.25892 handle_command_reply: tid=0 2024-09-12T03:03:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.287+0000 7ff09d418740 1 client.25892 shutdown 2024-09-12T03:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.291+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 msgr2=0x5597ff943820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.291+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff08c046000 tx=0x7ff08c003de0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 shutdown_connections 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff978010 0x5597ff97a480 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff943820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93b5e0 0x5597ff9432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 --2- 172.21.15.46:0/3186598076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93acc0 0x5597ff942da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 >> 172.21.15.46:0/3186598076 conn(0x5597ff937230 msgr2=0x5597ff93cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.293+0000 7ff0737fe640 0 client.25892 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.303+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 shutdown_connections 2024-09-12T03:03:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.303+0000 7ff09d418740 1 -- 172.21.15.46:0/3186598076 wait complete. 2024-09-12T03:03:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.303+0000 7ff09d418740 20 client.25892 trim_cache size 0 max 16384 2024-09-12T03:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.359+0000 7ff09d418740 1 Processor -- start 2024-09-12T03:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.359+0000 7ff09d418740 1 -- start start 2024-09-12T03:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff09d418740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff093fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff093fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45318/0 (socket says 172.21.15.46:45318) 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff09d418740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff936150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff093fff640 1 -- 172.21.15.46:0/3506445068 learned_addr learned my addr 172.21.15.46:0/3506445068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.360+0000 7ff09d418740 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff97fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.361+0000 7ff0937fe640 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff936150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.361+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff080012ad0 con 0x5597ff93acc0 2024-09-12T03:03:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.361+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff080010340 con 0x5597ff945e80 2024-09-12T03:03:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.361+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff08000e1b0 con 0x5597ff93b5e0 2024-09-12T03:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 -- 172.21.15.46:0/3506445068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 msgr2=0x5597ff936150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff936150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 -- 172.21.15.46:0/3506445068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 msgr2=0x5597ff97fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff97fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 -- 172.21.15.46:0/3506445068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ff976a50 con 0x5597ff93acc0 2024-09-12T03:03:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.365+0000 7ff093fff640 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff0880d30d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92698ef356ac4e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.379+0000 7ff079ffb640 1 -- 172.21.15.46:0/3506445068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08809dc30 con 0x5597ff93acc0 2024-09-12T03:03:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.379+0000 7ff079ffb640 1 -- 172.21.15.46:0/3506445068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08809e0f0 con 0x5597ff93acc0 2024-09-12T03:03:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.379+0000 7ff079ffb640 1 -- 172.21.15.46:0/3506445068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08809dec0 con 0x5597ff93acc0 2024-09-12T03:03:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.380+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 msgr2=0x5597ff943650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.380+0000 7ff09d418740 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff0880d30d0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.384+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 shutdown_connections 2024-09-12T03:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.384+0000 7ff09d418740 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff97fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.384+0000 7ff09d418740 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff936150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.385+0000 7ff09d418740 1 --2- 172.21.15.46:0/3506445068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff943650 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.385+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 >> 172.21.15.46:0/3506445068 conn(0x5597ff937230 msgr2=0x5597ff935a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.388+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 shutdown_connections 2024-09-12T03:03:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.388+0000 7ff09d418740 1 -- 172.21.15.46:0/3506445068 wait complete. 2024-09-12T03:03:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.388+0000 7ff09d418740 1 -- start start 2024-09-12T03:03:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.389+0000 7ff09d418740 1 Processor -- start 2024-09-12T03:03:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.389+0000 7ff09d418740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.389+0000 7ff09d418740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff09d418740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff940700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:57480/0 (socket says 172.21.15.46:57480) 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff0937fe640 1 -- 172.21.15.46:0/1900129831 learned_addr learned my addr 172.21.15.46:0/1900129831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff08000e800 con 0x5597ff93acc0 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff08000fc70 con 0x5597ff945e80 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff098b3c640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff940700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.390+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff08000eed0 con 0x5597ff93b5e0 2024-09-12T03:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff09d418740 10 client.0 _ll_register_callbacks cb 0x7ff08c012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff0937fe640 1 -- 172.21.15.46:0/1900129831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 msgr2=0x5597ff940700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff0937fe640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff940700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff0937fe640 1 -- 172.21.15.46:0/1900129831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 msgr2=0x5597ff93fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff0937fe640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.391+0000 7ff0937fe640 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ff9e1d60 con 0x5597ff93b5e0 2024-09-12T03:03:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.397+0000 7ff0937fe640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff094118000 tx=0x7ff08400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac4b79746d0a9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.397+0000 7ff0737fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:03:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.398+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400ee70 con 0x5597ff93b5e0 2024-09-12T03:03:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.398+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08809e0f0 con 0x5597ff93b5e0 2024-09-12T03:03:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.398+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7ff084013450 con 0x5597ff93b5e0 2024-09-12T03:03:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.401+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff084022050 con 0x5597ff93b5e0 2024-09-12T03:03:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.401+0000 7ff09d418740 10 client.25846 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.401+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597ff9e2fa0 con 0x5597ff93b5e0 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.405+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08809e0f0 con 0x5597ff93b5e0 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.405+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ff084014850 con 0x5597ff93b5e0 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.405+0000 7ff09d418740 10 client.25846 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.405+0000 7ff09d418740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.405+0000 7ff09d418740 20 client.25846 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.406+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff9e37e0 0x5597ff9e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.406+0000 7ff09d418740 4 client.25846 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:03:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.406+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597ff823900 con 0x5597ff9e37e0 2024-09-12T03:03:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.407+0000 7ff093fff640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff9e37e0 0x5597ff9e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.413+0000 7ff093fff640 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff9e37e0 0x5597ff9e5c00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba6738e8b44c15 server_cookie=78c48b257c062d76 in_seq=0 out_seq=0 2024-09-12T03:03:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.413+0000 7ff0737fe640 10 client.25846 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:18 smithi196 ceph-mon[34587]: pgmap v412: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 2.66k op/s 2024-09-12T03:03:18.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:18 smithi046 ceph-mon[30951]: pgmap v412: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 2.66k op/s 2024-09-12T03:03:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff0737fe640 1 -- 172.21.15.46:0/1900129831 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597ff823900 con 0x5597ff9e37e0 2024-09-12T03:03:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff0737fe640 10 client.25846 handle_command_reply: tid=0 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 client.25846 shutdown 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 msgr2=0x5597ff9401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff094118000 tx=0x7ff08400e5a0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 shutdown_connections 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5597ff9e37e0 0x5597ff9e5c00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597ff945e80 0x5597ff940700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597ff93b5e0 0x5597ff9401c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 --2- 172.21.15.46:0/1900129831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597ff93acc0 0x5597ff93fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 >> 172.21.15.46:0/1900129831 conn(0x5597ff937230 msgr2=0x5597ff936260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff0737fe640 0 client.25846 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.817+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 shutdown_connections 2024-09-12T03:03:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.818+0000 7ff09d418740 1 -- 172.21.15.46:0/1900129831 wait complete. 2024-09-12T03:03:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.818+0000 7ff09d418740 20 client.25846 trim_cache size 0 max 16384 2024-09-12T03:03:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:03:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:03:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:03:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 msgr2=0x7ff07c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 0x7ff07c0a53c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005e30 tx=0x7ff08c005ef0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 msgr2=0x7ff09411a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ea20 tx=0x7ff08800eee0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 shutdown_connections 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7ff07c0a2f00 0x7ff07c0a53c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff09411b140 0x7ff094116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0941095d0 0x7ff094116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 --2- 172.21.15.46:0/1677677561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0941091f0 0x7ff09411a7b0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ea20 tx=0x7ff08800eee0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.819+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 >> 172.21.15.46:0/1677677561 conn(0x7ff09406cbf0 msgr2=0x7ff094070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.822+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 shutdown_connections 2024-09-12T03:03:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:18.822+0000 7ff09933d640 1 -- 172.21.15.46:0/1677677561 wait complete. 2024-09-12T03:03:18.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:18 smithi145 ceph-mon[34674]: pgmap v412: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 2.66k op/s 2024-09-12T03:03:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:20 smithi196 ceph-mon[34587]: pgmap v413: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:20.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:20 smithi046 ceph-mon[30951]: pgmap v413: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:20 smithi145 ceph-mon[34674]: pgmap v413: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:22 smithi196 ceph-mon[34587]: pgmap v414: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-12T03:03:22.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:22 smithi046 ceph-mon[30951]: pgmap v414: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-12T03:03:22.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:22 smithi145 ceph-mon[34674]: pgmap v414: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 3.14k op/s 2024-09-12T03:03:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:24 smithi196 ceph-mon[34587]: pgmap v415: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:24 smithi046 ceph-mon[30951]: pgmap v415: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:24.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:24 smithi145 ceph-mon[34674]: pgmap v415: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-09-12T03:03:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:26 smithi196 ceph-mon[34587]: pgmap v416: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:26 smithi046 ceph-mon[30951]: pgmap v416: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:26 smithi145 ceph-mon[34674]: pgmap v416: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:28 smithi196 ceph-mon[34587]: pgmap v417: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:28.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:28 smithi046 ceph-mon[30951]: pgmap v417: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:28.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:28 smithi145 ceph-mon[34674]: pgmap v417: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-12T03:03:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:30 smithi196 ceph-mon[34587]: pgmap v418: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:30.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:30 smithi046 ceph-mon[30951]: pgmap v418: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:30 smithi145 ceph-mon[34674]: pgmap v418: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.066+0000 7fc8047e8640 1 -- 172.21.15.46:0/2691693587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40b76f0 msgr2=0x7fc7f40b9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.066+0000 7fc8047e8640 1 --2- 172.21.15.46:0/2691693587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40b76f0 0x7fc7f40b9ae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc065a40 tx=0x7fc7fc08e420 comp rx=0 tx=0).stop 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.067+0000 7fc8047e8640 1 -- 172.21.15.46:0/2691693587 shutdown_connections 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.067+0000 7fc8047e8640 1 --2- 172.21.15.46:0/2691693587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40b76f0 0x7fc7f40b9ae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.067+0000 7fc8047e8640 1 --2- 172.21.15.46:0/2691693587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7f40a52f0 0x7fc7f40b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.067+0000 7fc8047e8640 1 --2- 172.21.15.46:0/2691693587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40a49d0 0x7fc7f40a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.067+0000 7fc8047e8640 1 -- 172.21.15.46:0/2691693587 >> 172.21.15.46:0/2691693587 conn(0x7fc7f401a0d0 msgr2=0x7fc7f401a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.068+0000 7fc8047e8640 1 -- 172.21.15.46:0/2691693587 shutdown_connections 2024-09-12T03:03:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.072+0000 7fc8047e8640 1 -- 172.21.15.46:0/2691693587 wait complete. 2024-09-12T03:03:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.074+0000 7fc8047e8640 1 Processor -- start 2024-09-12T03:03:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.077+0000 7fc8047e8640 1 -- start start 2024-09-12T03:03:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.077+0000 7fc8047e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.078+0000 7fc8047e8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7f40a52f0 0x7fc7f41438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.078+0000 7fc8047e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40b76f0 0x7fc7f4147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.078+0000 7fc8047e8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40bced0 con 0x7fc7f40b76f0 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.078+0000 7fc8047e8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40bcd50 con 0x7fc7f40a52f0 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.078+0000 7fc8047e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40bd050 con 0x7fc7f40a49d0 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.079+0000 7fc802584640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.079+0000 7fc802584640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:53686/0 (socket says 172.21.15.46:53686) 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.079+0000 7fc802584640 1 -- 172.21.15.46:0/1549108531 learned_addr learned my addr 172.21.15.46:0/1549108531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.079+0000 7fc803586640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40b76f0 0x7fc7f4147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.080+0000 7fc802584640 1 -- 172.21.15.46:0/1549108531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7f40a52f0 msgr2=0x7fc7f41438a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:03:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.080+0000 7fc802584640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7f40a52f0 0x7fc7f41438a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.080+0000 7fc802584640 1 -- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40b76f0 msgr2=0x7fc7f4147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.080+0000 7fc802584640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40b76f0 0x7fc7f4147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.080+0000 7fc802584640 1 -- 172.21.15.46:0/1549108531 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f4148320 con 0x7fc7f40a49d0 2024-09-12T03:03:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.087+0000 7fc802584640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00d980 tx=0x7fc7ec00de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.088+0000 7fc7f37fe640 1 -- 172.21.15.46:0/1549108531 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec017070 con 0x7fc7f40a49d0 2024-09-12T03:03:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.089+0000 7fc7f37fe640 1 -- 172.21.15.46:0/1549108531 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7ec009c30 con 0x7fc7f40a49d0 2024-09-12T03:03:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.089+0000 7fc7f37fe640 1 -- 172.21.15.46:0/1549108531 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00f620 con 0x7fc7f40a49d0 2024-09-12T03:03:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.092+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f4148610 con 0x7fc7f40a49d0 2024-09-12T03:03:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.092+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f40ac990 con 0x7fc7f40a49d0 2024-09-12T03:03:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.099+0000 7fc807662740 1 Processor -- start 2024-09-12T03:03:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.102+0000 7fc7f37fe640 1 -- 172.21.15.46:0/1549108531 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc7ec010460 con 0x7fc7f40a49d0 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.106+0000 7fc807662740 1 -- start start 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861eddf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55861ed432f0 con 0x55861eddf5e0 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55861eceb270 con 0x55861ede9e80 2024-09-12T03:03:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.107+0000 7fc807662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55861ed3f2b0 con 0x55861eddecc0 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.108+0000 7fc801d83640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.108+0000 7fc801d83640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56434/0 (socket says 172.21.15.46:56434) 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.108+0000 7fc801d83640 1 -- 172.21.15.46:0/2296615085 learned_addr learned my addr 172.21.15.46:0/2296615085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.108+0000 7fc7f37fe640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 0x7fc7e80a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 -- 172.21.15.46:0/2296615085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 msgr2=0x55861eddf0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861eddf0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 -- 172.21.15.46:0/2296615085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 msgr2=0x55861ede9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc7f37fe640 1 -- 172.21.15.46:0/1549108531 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7fc7ec0cf260 con 0x7fc7f40a49d0 2024-09-12T03:03:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 -- 172.21.15.46:0/2296615085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55861ecea340 con 0x55861ede9e80 2024-09-12T03:03:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.109+0000 7fc801d83640 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009b70 tx=0x7fc7f800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2436f6009a81d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.111+0000 7fc7f0ff9640 1 -- 172.21.15.46:0/2296615085 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8009e60 con 0x55861ede9e80 2024-09-12T03:03:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.111+0000 7fc7f0ff9640 1 -- 172.21.15.46:0/2296615085 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f80054d0 con 0x55861ede9e80 2024-09-12T03:03:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.111+0000 7fc7f0ff9640 1 -- 172.21.15.46:0/2296615085 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8010490 con 0x55861ede9e80 2024-09-12T03:03:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.113+0000 7fc803586640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 0x7fc7e80a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.114+0000 7fc807662740 1 -- 172.21.15.46:0/2296615085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 msgr2=0x55861edec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.114+0000 7fc807662740 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009b70 tx=0x7fc7f800e720 comp rx=0 tx=0).stop 2024-09-12T03:03:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.116+0000 7fc807662740 1 -- 172.21.15.46:0/2296615085 shutdown_connections 2024-09-12T03:03:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.116+0000 7fc807662740 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861edec270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.116+0000 7fc807662740 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.116+0000 7fc807662740 1 --2- 172.21.15.46:0/2296615085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861eddf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.117+0000 7fc807662740 1 -- 172.21.15.46:0/2296615085 >> 172.21.15.46:0/2296615085 conn(0x55861eddb230 msgr2=0x55861eddb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.121+0000 7fc807662740 1 -- 172.21.15.46:0/2296615085 shutdown_connections 2024-09-12T03:03:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.125+0000 7fc803586640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 0x7fc7e80a5340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02dd50 tx=0x7fc7fc068340 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.125+0000 7fc807662740 1 -- 172.21.15.46:0/2296615085 wait complete. 2024-09-12T03:03:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.129+0000 7fc807662740 1 -- start start 2024-09-12T03:03:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.130+0000 7fc807662740 1 Processor -- start 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddecc0 0x55861ede6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 0x55861ede7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55861edeca50 con 0x55861eddf5e0 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55861eccc290 con 0x55861eddecc0 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55861edecbd0 con 0x55861ede9e80 2024-09-12T03:03:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.132+0000 7fc807662740 10 client.0 _ll_register_callbacks cb 0x55861ecfd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.133+0000 7fc801d83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.133+0000 7fc801d83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41584/0 (socket says 172.21.15.46:41584) 2024-09-12T03:03:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.133+0000 7fc801d83640 1 -- 172.21.15.46:0/3706055746 learned_addr learned my addr 172.21.15.46:0/3706055746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.134+0000 7fc803586640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 0x55861ede7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.135+0000 7fc803586640 1 -- 172.21.15.46:0/3706055746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddecc0 msgr2=0x55861ede6d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.135+0000 7fc803586640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddecc0 0x55861ede6d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.135+0000 7fc803586640 1 -- 172.21.15.46:0/3706055746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 msgr2=0x55861ede72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.135+0000 7fc803586640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.135+0000 7fc803586640 1 -- 172.21.15.46:0/3706055746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861ea56210 con 0x55861ede9e80 2024-09-12T03:03:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.136+0000 7fc803586640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 0x55861ede7800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc065a40 tx=0x7fc7fc0505c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf837371a04ceab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.138+0000 7fc7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:03:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.138+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0b4060 con 0x55861ede9e80 2024-09-12T03:03:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.138+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f80054d0 con 0x55861ede9e80 2024-09-12T03:03:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.138+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7fc7fc06a270 con 0x55861ede9e80 2024-09-12T03:03:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.139+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc062e50 con 0x55861ede9e80 2024-09-12T03:03:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.142+0000 7fc807662740 10 client.25855 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.142+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55861eb22620 con 0x55861ede9e80 2024-09-12T03:03:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.157+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f80054d0 con 0x55861ede9e80 2024-09-12T03:03:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.157+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc7fc0a1040 con 0x55861ede9e80 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 10 client.25855 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 20 client.25855 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861edf9e70 0x55861edfc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 4 client.25855 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.158+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55861ed50a20 con 0x55861edf9e70 2024-09-12T03:03:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.162+0000 7fc802584640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861edf9e70 0x55861edfc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.171+0000 7fc802584640 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861edf9e70 0x55861edfc2a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff35862c2a137 server_cookie=80f189565f1b04aa in_seq=0 out_seq=0 2024-09-12T03:03:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.171+0000 7fc7e5ffb640 10 client.25855 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.174+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/3706055746 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55861ed50a20 con 0x55861edf9e70 2024-09-12T03:03:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.174+0000 7fc7e5ffb640 10 client.25855 handle_command_reply: tid=0 2024-09-12T03:03:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.176+0000 7fc807662740 1 client.25855 shutdown 2024-09-12T03:03:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.178+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 msgr2=0x55861ede7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.178+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 0x55861ede7800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc065a40 tx=0x7fc7fc0505c0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 shutdown_connections 2024-09-12T03:03:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861edf9e70 0x55861edfc2a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861ede9e80 0x55861ede7800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861ede72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 --2- 172.21.15.46:0/3706055746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddecc0 0x55861ede6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.179+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 >> 172.21.15.46:0/3706055746 conn(0x55861eddb230 msgr2=0x55861ede0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.180+0000 7fc7e5ffb640 0 client.25855 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.186+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 shutdown_connections 2024-09-12T03:03:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.186+0000 7fc807662740 1 -- 172.21.15.46:0/3706055746 wait complete. 2024-09-12T03:03:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.189+0000 7fc807662740 20 client.25855 trim_cache size 0 max 16384 2024-09-12T03:03:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.250+0000 7fc807662740 1 Processor -- start 2024-09-12T03:03:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.250+0000 7fc807662740 1 -- start start 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.251+0000 7fc807662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861ede7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.251+0000 7fc807662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.251+0000 7fc807662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861ede9e80 0x55861ee020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.251+0000 7fc801d83640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.251+0000 7fc801d83640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:56448/0 (socket says 172.21.15.46:56448) 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.252+0000 7fc801d83640 1 -- 172.21.15.46:0/14308559 learned_addr learned my addr 172.21.15.46:0/14308559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.252+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41652d0 con 0x55861ede9e80 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.252+0000 7fc803586640 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861ede9e80 0x55861ee020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.252+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4162b40 con 0x55861eddf5e0 2024-09-12T03:03:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.252+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40a4db0 con 0x55861eddecc0 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 -- 172.21.15.46:0/14308559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 msgr2=0x55861ede7630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861ede7630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 -- 172.21.15.46:0/14308559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861ede9e80 msgr2=0x55861ee020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861ede9e80 0x55861ee020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 -- 172.21.15.46:0/14308559 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55861edec5d0 con 0x55861eddf5e0 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc801d83640 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009230 tx=0x7fc7f8009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f5be91b7919db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.255+0000 7fc7e7fff640 1 -- 172.21.15.46:0/14308559 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f801d040 con 0x55861eddf5e0 2024-09-12T03:03:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.256+0000 7fc7e7fff640 1 -- 172.21.15.46:0/14308559 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f8009e60 con 0x55861eddf5e0 2024-09-12T03:03:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.256+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 msgr2=0x55861ee01bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.256+0000 7fc807662740 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009230 tx=0x7fc7f8009690 comp rx=0 tx=0).stop 2024-09-12T03:03:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.261+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 shutdown_connections 2024-09-12T03:03:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.261+0000 7fc807662740 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861ede9e80 0x55861ee020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.261+0000 7fc807662740 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861eddf5e0 0x55861ee01bb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.261+0000 7fc807662740 1 --2- 172.21.15.46:0/14308559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861ede7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.261+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 >> 172.21.15.46:0/14308559 conn(0x55861eddb230 msgr2=0x55861ede2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.263+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 shutdown_connections 2024-09-12T03:03:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.263+0000 7fc807662740 1 -- 172.21.15.46:0/14308559 wait complete. 2024-09-12T03:03:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.264+0000 7fc807662740 1 -- start start 2024-09-12T03:03:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.264+0000 7fc807662740 1 Processor -- start 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.264+0000 7fc807662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.264+0000 7fc807662740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861edf2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.264+0000 7fc807662740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861ede33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc807662740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4161020 con 0x55861eddf5e0 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc807662740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4162470 con 0x55861ede9e80 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc802584640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc802584640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:53718/0 (socket says 172.21.15.46:53718) 2024-09-12T03:03:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc802584640 1 -- 172.21.15.46:0/1621064831 learned_addr learned my addr 172.21.15.46:0/1621064831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc807662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41616d0 con 0x55861eddecc0 2024-09-12T03:03:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.265+0000 7fc807662740 10 client.0 _ll_register_callbacks cb 0x7fc7f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.266+0000 7fc803586640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861ede33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.266+0000 7fc801d83640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861edf2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.266+0000 7fc802584640 1 -- 172.21.15.46:0/1621064831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 msgr2=0x55861ede33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.266+0000 7fc802584640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861ede33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.266+0000 7fc802584640 1 -- 172.21.15.46:0/1621064831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 msgr2=0x55861edf2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.267+0000 7fc802584640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861edf2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.267+0000 7fc802584640 1 -- 172.21.15.46:0/1621064831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861ede3900 con 0x55861eddecc0 2024-09-12T03:03:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.267+0000 7fc802584640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0977f0 tx=0x7fc7ec005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e881bef31a3b4bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.267+0000 7fc7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:03:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.267+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec09b390 con 0x55861eddecc0 2024-09-12T03:03:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.268+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f8009e60 con 0x55861eddecc0 2024-09-12T03:03:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.268+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9555+0+0 (secure 0 0 0) 0x7fc7ec0b4260 con 0x55861eddecc0 2024-09-12T03:03:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.268+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0bb050 con 0x55861eddecc0 2024-09-12T03:03:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.270+0000 7fc807662740 10 client.25858 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.270+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55861ee64370 con 0x55861eddecc0 2024-09-12T03:03:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.275+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f8009e60 con 0x55861eddecc0 2024-09-12T03:03:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.275+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc7ec0a6020 con 0x55861eddecc0 2024-09-12T03:03:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 10 client.25858 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 10 client.25858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 20 client.25858 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861ee64a50 0x55861ee66e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 4 client.25858 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:03:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.279+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55861ed4ad70 con 0x55861ee64a50 2024-09-12T03:03:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.280+0000 7fc801d83640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861ee64a50 0x55861ee66e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.285+0000 7fc801d83640 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861ee64a50 0x55861ee66e70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95abd5aab204a980 server_cookie=756e3d4f5b3f282e in_seq=0 out_seq=0 2024-09-12T03:03:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.285+0000 7fc7e5ffb640 10 client.25858 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.534+0000 7fc7e5ffb640 1 -- 172.21.15.46:0/1621064831 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55861ed4ad70 con 0x55861ee64a50 2024-09-12T03:03:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.534+0000 7fc7e5ffb640 10 client.25858 handle_command_reply: tid=0 2024-09-12T03:03:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.534+0000 7fc807662740 1 client.25858 shutdown 2024-09-12T03:03:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.539+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 msgr2=0x55861edf1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.539+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0977f0 tx=0x7fc7ec005bb0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 shutdown_connections 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55861ee64a50 0x55861ee66e70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55861ede9e80 0x55861ede33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55861eddf5e0 0x55861edf2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 --2- 172.21.15.46:0/1621064831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55861eddecc0 0x55861edf1ee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.541+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 >> 172.21.15.46:0/1621064831 conn(0x55861eddb230 msgr2=0x55861edd9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.542+0000 7fc7e5ffb640 0 client.25858 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.542+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 shutdown_connections 2024-09-12T03:03:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.544+0000 7fc807662740 1 -- 172.21.15.46:0/1621064831 wait complete. 2024-09-12T03:03:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.544+0000 7fc807662740 20 client.25858 trim_cache size 0 max 16384 2024-09-12T03:03:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:03:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:03:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:03:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:03:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.549+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 msgr2=0x7fc7e80a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.549+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 0x7fc7e80a5340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02dd50 tx=0x7fc7fc068340 comp rx=0 tx=0).stop 2024-09-12T03:03:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.549+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 msgr2=0x7fc7f4143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.549+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00d980 tx=0x7fc7ec00de40 comp rx=0 tx=0).stop 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.551+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 shutdown_connections 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.551+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc7e80a2e80 0x7fc7e80a5340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.551+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f40b76f0 0x7fc7f4147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.551+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7f40a52f0 0x7fc7f41438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.551+0000 7fc8047e8640 1 --2- 172.21.15.46:0/1549108531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f40a49d0 0x7fc7f4143360 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.552+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 >> 172.21.15.46:0/1549108531 conn(0x7fc7f401a0d0 msgr2=0x7fc7f40b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.563+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 shutdown_connections 2024-09-12T03:03:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:32.563+0000 7fc8047e8640 1 -- 172.21.15.46:0/1549108531 wait complete. 2024-09-12T03:03:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:32 smithi196 ceph-mon[34587]: pgmap v419: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:03:32.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:32 smithi046 ceph-mon[30951]: pgmap v419: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:03:32.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:32 smithi145 ceph-mon[34674]: pgmap v419: 129 pgs: 16 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 76 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:03:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:34 smithi196 ceph-mon[34587]: pgmap v420: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-09-12T03:03:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:34 smithi046 ceph-mon[30951]: pgmap v420: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-09-12T03:03:34.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:34 smithi145 ceph-mon[34674]: pgmap v420: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-09-12T03:03:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:36 smithi196 ceph-mon[34587]: pgmap v421: 129 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 92 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-12T03:03:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:36 smithi046 ceph-mon[30951]: pgmap v421: 129 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 92 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-12T03:03:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:36 smithi145 ceph-mon[34674]: pgmap v421: 129 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 92 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-12T03:03:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:38 smithi196 ceph-mon[34587]: pgmap v422: 129 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 99 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:38.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:38 smithi046 ceph-mon[30951]: pgmap v422: 129 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 99 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:38.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:38 smithi145 ceph-mon[34674]: pgmap v422: 129 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 99 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-12T03:03:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:40 smithi196 ceph-mon[34587]: pgmap v423: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-12T03:03:40.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:40 smithi046 ceph-mon[30951]: pgmap v423: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-12T03:03:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:40 smithi145 ceph-mon[34674]: pgmap v423: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-12T03:03:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:42 smithi196 ceph-mon[34587]: pgmap v424: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-12T03:03:42.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:42 smithi046 ceph-mon[30951]: pgmap v424: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-12T03:03:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:42 smithi145 ceph-mon[34674]: pgmap v424: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-09-12T03:03:43.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:43 smithi196 ceph-mon[34587]: pgmap v425: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:03:43.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:43 smithi046 ceph-mon[30951]: pgmap v425: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:03:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:43 smithi145 ceph-mon[34674]: pgmap v425: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-12T03:03:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.815+0000 7fba6939f640 1 -- 172.21.15.46:0/3932320163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 msgr2=0x7fba6411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.815+0000 7fba6939f640 1 --2- 172.21.15.46:0/3932320163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 0x7fba6411d530 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fba54009900 tx=0x7fba5402a440 comp rx=0 tx=0).stop 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.816+0000 7fba6939f640 1 -- 172.21.15.46:0/3932320163 shutdown_connections 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.816+0000 7fba6939f640 1 --2- 172.21.15.46:0/3932320163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 0x7fba6411d530 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.816+0000 7fba6939f640 1 --2- 172.21.15.46:0/3932320163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641095d0 0x7fba64109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.816+0000 7fba6939f640 1 --2- 172.21.15.46:0/3932320163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641091f0 0x7fba64110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.816+0000 7fba6939f640 1 -- 172.21.15.46:0/3932320163 >> 172.21.15.46:0/3932320163 conn(0x7fba6406cbf0 msgr2=0x7fba6406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.833+0000 7fba6939f640 1 -- 172.21.15.46:0/3932320163 shutdown_connections 2024-09-12T03:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.833+0000 7fba6939f640 1 -- 172.21.15.46:0/3932320163 wait complete. 2024-09-12T03:03:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.834+0000 7fba6939f640 1 Processor -- start 2024-09-12T03:03:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.834+0000 7fba6939f640 1 -- start start 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.834+0000 7fba6939f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 0x7fba641a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.834+0000 7fba6939f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 0x7fba641a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 0x7fba641a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:34180/0 (socket says 172.21.15.46:34180) 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba6939f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 0x7fba641ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba6939f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba64120a70 con 0x7fba6411b140 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba6939f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba641208f0 con 0x7fba641091f0 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba6939f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba64120bf0 con 0x7fba641095d0 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:39854/0 (socket says 172.21.15.46:39854) 2024-09-12T03:03:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.835+0000 7fba637fe640 1 -- 172.21.15.46:0/1535921738 learned_addr learned my addr 172.21.15.46:0/1535921738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.836+0000 7fba637fe640 1 -- 172.21.15.46:0/1535921738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 msgr2=0x7fba641a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.836+0000 7fba637fe640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 0x7fba641a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.836+0000 7fba637fe640 1 -- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 msgr2=0x7fba641ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.836+0000 7fba637fe640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 0x7fba641ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.836+0000 7fba637fe640 1 -- 172.21.15.46:0/1535921738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641ac070 con 0x7fba641095d0 2024-09-12T03:03:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.838+0000 7fba637fe640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba5800b450 tx=0x7fba5800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.839+0000 7fba617fa640 1 -- 172.21.15.46:0/1535921738 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58015020 con 0x7fba641095d0 2024-09-12T03:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.839+0000 7fba617fa640 1 -- 172.21.15.46:0/1535921738 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba58011d40 con 0x7fba641095d0 2024-09-12T03:03:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.839+0000 7fba617fa640 1 -- 172.21.15.46:0/1535921738 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58010e30 con 0x7fba641095d0 2024-09-12T03:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.839+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba641ac360 con 0x7fba641095d0 2024-09-12T03:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.840+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641b3bf0 con 0x7fba641095d0 2024-09-12T03:03:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.843+0000 7fba6d472740 1 Processor -- start 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.843+0000 7fba6d472740 1 -- start start 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 0x55b5bb6950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb5f92f0 con 0x55b5bb6955e0 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb5a1270 con 0x55b5bb69fe80 2024-09-12T03:03:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba6d472740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb5f52b0 con 0x55b5bb694cc0 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba637fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba63fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba63fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48184/0 (socket says 172.21.15.46:48184) 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba63fff640 1 -- 172.21.15.46:0/579518711 learned_addr learned my addr 172.21.15.46:0/579518711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.844+0000 7fba68b9e640 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 0x55b5bb6950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.845+0000 7fba68b9e640 1 -- 172.21.15.46:0/579518711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb69fe80 msgr2=0x55b5bb6a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.845+0000 7fba68b9e640 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.845+0000 7fba68b9e640 1 -- 172.21.15.46:0/579518711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 msgr2=0x55b5bb69f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.845+0000 7fba68b9e640 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.845+0000 7fba68b9e640 1 -- 172.21.15.46:0/579518711 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5bb5a0340 con 0x55b5bb694cc0 2024-09-12T03:03:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.847+0000 7fba68b9e640 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 0x55b5bb6950a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba54009230 tx=0x7fba54004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d1ea204f7d5a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.848+0000 7fba617fa640 1 -- 172.21.15.46:0/1535921738 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba58018020 con 0x7fba641095d0 2024-09-12T03:03:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.849+0000 7fba617fa640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 0x7fba4c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.849+0000 7fba617fa640 1 -- 172.21.15.46:0/1535921738 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7fba580d6030 con 0x7fba641095d0 2024-09-12T03:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.850+0000 7fba4a7fc640 1 -- 172.21.15.46:0/579518711 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5403e070 con 0x55b5bb694cc0 2024-09-12T03:03:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.850+0000 7fba4a7fc640 1 -- 172.21.15.46:0/579518711 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba54005500 con 0x55b5bb694cc0 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.851+0000 7fba6d472740 1 -- 172.21.15.46:0/579518711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 msgr2=0x55b5bb6950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.851+0000 7fba6d472740 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 0x55b5bb6950a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba54009230 tx=0x7fba54004180 comp rx=0 tx=0).stop 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.851+0000 7fba6d472740 1 -- 172.21.15.46:0/579518711 shutdown_connections 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.851+0000 7fba6d472740 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.851+0000 7fba6d472740 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.852+0000 7fba6d472740 1 --2- 172.21.15.46:0/579518711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb694cc0 0x55b5bb6950a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.852+0000 7fba6d472740 1 -- 172.21.15.46:0/579518711 >> 172.21.15.46:0/579518711 conn(0x55b5bb691230 msgr2=0x55b5bb691610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.852+0000 7fba63fff640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 0x7fba4c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.853+0000 7fba63fff640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 0x7fba4c0a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b5bb692090 tx=0x7fba5c00e040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.853+0000 7fba6d472740 1 -- 172.21.15.46:0/579518711 shutdown_connections 2024-09-12T03:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.853+0000 7fba6d472740 1 -- 172.21.15.46:0/579518711 wait complete. 2024-09-12T03:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.854+0000 7fba6d472740 1 -- start start 2024-09-12T03:03:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.854+0000 7fba6d472740 1 Processor -- start 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb69d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb6a2aa0 con 0x55b5bb6955e0 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb582290 con 0x55b5bb694cc0 2024-09-12T03:03:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5bb6a2c20 con 0x55b5bb69fe80 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba6d472740 10 client.0 _ll_register_callbacks cb 0x55b5bb5b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:34202/0 (socket says 172.21.15.46:34202) 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.855+0000 7fba63fff640 1 -- 172.21.15.46:0/3934178679 learned_addr learned my addr 172.21.15.46:0/3934178679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba637fe640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba63fff640 1 -- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 msgr2=0x55b5bb69d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba63fff640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb69d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba63fff640 1 -- 172.21.15.46:0/3934178679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 msgr2=0x55b5bb69d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba63fff640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.856+0000 7fba63fff640 1 -- 172.21.15.46:0/3934178679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5bb30c210 con 0x55b5bb694cc0 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.857+0000 7fba63fff640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00af00 tx=0x7fba5c004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6f0d617ff419bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.857+0000 7fba437fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.865+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0215f0 con 0x55b5bb694cc0 2024-09-12T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.865+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7fba5c021790 con 0x55b5bb694cc0 2024-09-12T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.865+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba54005500 con 0x55b5bb694cc0 2024-09-12T03:03:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.870+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba5c021990 con 0x55b5bb694cc0 2024-09-12T03:03:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.871+0000 7fba6d472740 10 client.25916 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.871+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b5bb3d8620 con 0x55b5bb694cc0 2024-09-12T03:03:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.877+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba54005500 con 0x55b5bb694cc0 2024-09-12T03:03:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.877+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fba5c016ca0 con 0x55b5bb694cc0 2024-09-12T03:03:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 10 client.25916 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 20 client.25916 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6c12d0 0x55b5bb6c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 4 client.25916 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.878+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5bb606a20 con 0x55b5bb6c12d0 2024-09-12T03:03:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.881+0000 7fba637fe640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6c12d0 0x55b5bb6c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.882+0000 7fba637fe640 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6c12d0 0x55b5bb6c3790 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6802a7ad6cc00f2 server_cookie=7312eec2f3a88ba6 in_seq=0 out_seq=0 2024-09-12T03:03:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.882+0000 7fba437fe640 10 client.25916 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.886+0000 7fba437fe640 1 -- 172.21.15.46:0/3934178679 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5bb606a20 con 0x55b5bb6c12d0 2024-09-12T03:03:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.886+0000 7fba437fe640 10 client.25916 handle_command_reply: tid=0 2024-09-12T03:03:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.886+0000 7fba6d472740 1 client.25916 shutdown 2024-09-12T03:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.901+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 msgr2=0x55b5bb69ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.901+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00af00 tx=0x7fba5c004a80 comp rx=0 tx=0).stop 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 shutdown_connections 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6c12d0 0x55b5bb6c3790 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb69d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb6955e0 0x55b5bb69d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 --2- 172.21.15.46:0/3934178679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb69ccb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.905+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 >> 172.21.15.46:0/3934178679 conn(0x55b5bb691230 msgr2=0x55b5bb696fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.907+0000 7fba437fe640 0 client.25916 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.913+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 shutdown_connections 2024-09-12T03:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.914+0000 7fba6d472740 1 -- 172.21.15.46:0/3934178679 wait complete. 2024-09-12T03:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.916+0000 7fba6d472740 20 client.25916 trim_cache size 0 max 16384 2024-09-12T03:03:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.974+0000 7fba6d472740 1 Processor -- start 2024-09-12T03:03:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.975+0000 7fba6d472740 1 -- start start 2024-09-12T03:03:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 0x55b5bb6c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba63fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba63fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba63fff640 1 -- 172.21.15.46:0/4261778739 learned_addr learned my addr 172.21.15.46:0/4261778739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb6c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba641b7f40 con 0x55b5bb694cc0 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba641b57b0 con 0x55b5bb6955e0 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba64118900 con 0x55b5bb69fe80 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.976+0000 7fba637fe640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 0x55b5bb6c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.977+0000 7fba68b9e640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb6c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 -- 172.21.15.46:0/4261778739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 msgr2=0x55b5bb6c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb6c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 -- 172.21.15.46:0/4261778739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 msgr2=0x55b5bb6c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 0x55b5bb6c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 -- 172.21.15.46:0/4261778739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5bb6a25b0 con 0x55b5bb694cc0 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba637fe640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 0x55b5bb6c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.978+0000 7fba63fff640 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00ce70 tx=0x7fba5c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baaaee956da7737 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.979+0000 7fba49ffb640 1 -- 172.21.15.46:0/4261778739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c02a5f0 con 0x55b5bb694cc0 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.979+0000 7fba49ffb640 1 -- 172.21.15.46:0/4261778739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba5c026070 con 0x55b5bb694cc0 2024-09-12T03:03:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.979+0000 7fba49ffb640 1 -- 172.21.15.46:0/4261778739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c02ae00 con 0x55b5bb694cc0 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.980+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 msgr2=0x55b5bb69d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.980+0000 7fba6d472740 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00ce70 tx=0x7fba5c004af0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.980+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 shutdown_connections 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.980+0000 7fba6d472740 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb69fe80 0x55b5bb6c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.980+0000 7fba6d472740 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb6955e0 0x55b5bb6c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.981+0000 7fba6d472740 1 --2- 172.21.15.46:0/4261778739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb694cc0 0x55b5bb69d560 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.981+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 >> 172.21.15.46:0/4261778739 conn(0x55b5bb691230 msgr2=0x55b5bb69bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.981+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 shutdown_connections 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.981+0000 7fba6d472740 1 -- 172.21.15.46:0/4261778739 wait complete. 2024-09-12T03:03:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.981+0000 7fba6d472740 1 -- start start 2024-09-12T03:03:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.982+0000 7fba6d472740 1 Processor -- start 2024-09-12T03:03:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.982+0000 7fba6d472740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.983+0000 7fba6d472740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb6955e0 0x55b5bb6a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.983+0000 7fba6d472740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.983+0000 7fba6d472740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba64118f70 con 0x55b5bb69fe80 2024-09-12T03:03:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.983+0000 7fba6d472740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba641b50e0 con 0x55b5bb694cc0 2024-09-12T03:03:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.983+0000 7fba6d472740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba641b4340 con 0x55b5bb6955e0 2024-09-12T03:03:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.984+0000 7fba6d472740 10 client.0 _ll_register_callbacks cb 0x7fba5c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.984+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.984+0000 7fba63fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:34228/0 (socket says 172.21.15.46:34228) 2024-09-12T03:03:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.984+0000 7fba63fff640 1 -- 172.21.15.46:0/298013937 learned_addr learned my addr 172.21.15.46:0/298013937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.984+0000 7fba637fe640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb6955e0 0x55b5bb6a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 -- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb6955e0 msgr2=0x55b5bb6a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb6955e0 0x55b5bb6a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 -- 172.21.15.46:0/298013937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb69fe80 msgr2=0x55b5bb6a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 -- 172.21.15.46:0/298013937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5bb6a82b0 con 0x55b5bb694cc0 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.985+0000 7fba63fff640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fba5c037040 tx=0x7fba5c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df2277b1235032a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.992+0000 7fba437fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.992+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c003c40 con 0x55b5bb694cc0 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.992+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7fba5c003de0 con 0x55b5bb694cc0 2024-09-12T03:03:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.992+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba5c026070 con 0x55b5bb694cc0 2024-09-12T03:03:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.994+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba5c02a5f0 con 0x55b5bb694cc0 2024-09-12T03:03:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.995+0000 7fba6d472740 10 client.25922 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.996+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b5bb6b3870 con 0x55b5bb694cc0 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.996+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba5c026070 con 0x55b5bb694cc0 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.996+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fba5c004410 con 0x55b5bb694cc0 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 10 client.25922 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 20 client.25922 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6b3fe0 0x55b5bb6b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 4 client.25922 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:03:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.997+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5bb600d70 con 0x55b5bb6b3fe0 2024-09-12T03:03:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:45.999+0000 7fba637fe640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6b3fe0 0x55b5bb6b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.010+0000 7fba637fe640 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6b3fe0 0x55b5bb6b6400 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713beb41e4b68379 server_cookie=999aeaef2d02e0d6 in_seq=0 out_seq=0 2024-09-12T03:03:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.012+0000 7fba437fe640 10 client.25922 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.448+0000 7fba437fe640 1 -- 172.21.15.46:0/298013937 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5bb600d70 con 0x55b5bb6b3fe0 2024-09-12T03:03:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.448+0000 7fba437fe640 10 client.25922 handle_command_reply: tid=0 2024-09-12T03:03:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.448+0000 7fba6d472740 1 client.25922 shutdown 2024-09-12T03:03:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.449+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 msgr2=0x55b5bb6a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.449+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fba5c037040 tx=0x7fba5c009f50 comp rx=0 tx=0).stop 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 shutdown_connections 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55b5bb6b3fe0 0x55b5bb6b6400 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5bb69fe80 0x55b5bb6a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5bb6955e0 0x55b5bb6a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 --2- 172.21.15.46:0/298013937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b5bb694cc0 0x55b5bb6a4590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.458+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 >> 172.21.15.46:0/298013937 conn(0x55b5bb691230 msgr2=0x55b5bb69d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.459+0000 7fba437fe640 0 client.25922 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.462+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 shutdown_connections 2024-09-12T03:03:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.464+0000 7fba6d472740 1 -- 172.21.15.46:0/298013937 wait complete. 2024-09-12T03:03:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.464+0000 7fba6d472740 20 client.25922 trim_cache size 0 max 16384 2024-09-12T03:03:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:03:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:03:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:03:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:03:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.465+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 msgr2=0x7fba4c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.465+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 0x7fba4c0a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b5bb692090 tx=0x7fba5c00e040 comp rx=0 tx=0).stop 2024-09-12T03:03:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.465+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 msgr2=0x7fba641a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.465+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba5800b450 tx=0x7fba5800b910 comp rx=0 tx=0).stop 2024-09-12T03:03:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 shutdown_connections 2024-09-12T03:03:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fba4c0a2f00 0x7fba4c0a53c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba6411b140 0x7fba641ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba641095d0 0x7fba641a75c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 --2- 172.21.15.46:0/1535921738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba641091f0 0x7fba641a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.469+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 >> 172.21.15.46:0/1535921738 conn(0x7fba6406cbf0 msgr2=0x7fba6411b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.473+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 shutdown_connections 2024-09-12T03:03:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:46.475+0000 7fba6939f640 1 -- 172.21.15.46:0/1535921738 wait complete. 2024-09-12T03:03:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:46 smithi145 ceph-mon[34674]: pgmap v426: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.14k op/s 2024-09-12T03:03:46.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:46 smithi145 ceph-mon[34674]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:03:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:46 smithi196 ceph-mon[34587]: pgmap v426: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.14k op/s 2024-09-12T03:03:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:46 smithi196 ceph-mon[34587]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:03:46.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:46 smithi046 ceph-mon[30951]: pgmap v426: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 3.14k op/s 2024-09-12T03:03:46.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:46 smithi046 ceph-mon[30951]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:03:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:48 smithi145 ceph-mon[34674]: pgmap v428: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:03:48.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:48 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:48 smithi196 ceph-mon[34587]: pgmap v428: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:03:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:48 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:48.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:48 smithi046 ceph-mon[30951]: pgmap v428: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-12T03:03:48.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:48 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:50 smithi196 ceph-mon[34587]: pgmap v429: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:03:50.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:50 smithi046 ceph-mon[30951]: pgmap v429: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:03:50.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:50 smithi145 ceph-mon[34674]: pgmap v429: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2024-09-12T03:03:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:52 smithi196 ceph-mon[34587]: pgmap v430: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:03:52.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:52 smithi046 ceph-mon[30951]: pgmap v430: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:03:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:52 smithi145 ceph-mon[34674]: pgmap v430: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:03:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:54 smithi196 ceph-mon[34587]: pgmap v431: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-12T03:03:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:54 smithi046 ceph-mon[30951]: pgmap v431: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-12T03:03:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:54 smithi145 ceph-mon[34674]: pgmap v431: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-12T03:03:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:56 smithi196 ceph-mon[34587]: pgmap v432: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:03:56.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:56 smithi046 ceph-mon[30951]: pgmap v432: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:03:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:56 smithi145 ceph-mon[34674]: pgmap v432: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-09-12T03:03:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:57 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:57 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:57.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:57 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:03:57.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:57 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:57.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:57 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:57.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:57 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:03:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:57 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:57 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:57.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:57 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:03:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:03:58 smithi196 ceph-mon[34587]: pgmap v433: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:58.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:03:58 smithi046 ceph-mon[30951]: pgmap v433: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:03:58 smithi145 ceph-mon[34674]: pgmap v433: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-09-12T03:03:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.674+0000 7f29742f8640 1 -- 172.21.15.46:0/42947836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 msgr2=0x7f296c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.674+0000 7f29742f8640 1 --2- 172.21.15.46:0/42947836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c10d8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2964009be0 tx=0x7f296402a710 comp rx=0 tx=0).stop 2024-09-12T03:03:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.675+0000 7f29742f8640 1 -- 172.21.15.46:0/42947836 shutdown_connections 2024-09-12T03:03:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.675+0000 7f29742f8640 1 --2- 172.21.15.46:0/42947836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c10d8a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.675+0000 7f29742f8640 1 --2- 172.21.15.46:0/42947836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1009d0 0x7f296c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.675+0000 7f29742f8640 1 --2- 172.21.15.46:0/42947836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1069e0 0x7f296c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.676+0000 7f29742f8640 1 -- 172.21.15.46:0/42947836 >> 172.21.15.46:0/42947836 conn(0x7f296c0fc6b0 msgr2=0x7f296c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.682+0000 7f29742f8640 1 -- 172.21.15.46:0/42947836 shutdown_connections 2024-09-12T03:03:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.685+0000 7f29742f8640 1 -- 172.21.15.46:0/42947836 wait complete. 2024-09-12T03:03:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.686+0000 7f29742f8640 1 Processor -- start 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.691+0000 7f29742f8640 1 -- start start 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 0x7f296c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 0x7f296c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110d20 con 0x7f296c1069e0 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110ba0 con 0x7f296c101370 2024-09-12T03:03:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.692+0000 7f29742f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f296c110ea0 con 0x7f296c1009d0 2024-09-12T03:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.694+0000 7f2972895640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 0x7f296c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.694+0000 7f2972895640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 0x7f296c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39568/0 (socket says 172.21.15.46:39568) 2024-09-12T03:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.694+0000 7f2972895640 1 -- 172.21.15.46:0/1874788517 learned_addr learned my addr 172.21.15.46:0/1874788517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.694+0000 7f2971893640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.696+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 0x7f296c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.699+0000 7f2972094640 1 -- 172.21.15.46:0/1874788517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 msgr2=0x7f296c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.699+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.699+0000 7f2972094640 1 -- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 msgr2=0x7f296c1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.699+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 0x7f296c1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.700+0000 7f2972094640 1 -- 172.21.15.46:0/1874788517 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c196730 con 0x7f296c1009d0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.702+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 0x7f296c19acf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fd0 tx=0x7f296800b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.705+0000 7f29637fe640 1 -- 172.21.15.46:0/1874788517 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968015020 con 0x7f296c1009d0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.705+0000 7f29637fe640 1 -- 172.21.15.46:0/1874788517 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2968011ce0 con 0x7f296c1009d0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.705+0000 7f29637fe640 1 -- 172.21.15.46:0/1874788517 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968010de0 con 0x7f296c1009d0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.706+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c196a20 con 0x7f296c1009d0 2024-09-12T03:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.706+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c196f20 con 0x7f296c1009d0 2024-09-12T03:03:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 Processor -- start 2024-09-12T03:03:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 -- start start 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7088ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d7093770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596d6fed2f0 con 0x5596d7088af0 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5596d6f95270 con 0x5596d7093cb0 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.711+0000 7f2977095740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596d6fe92b0 con 0x5596d7089410 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.712+0000 7f2971893640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.712+0000 7f2971893640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:52376/0 (socket says 172.21.15.46:52376) 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.712+0000 7f2971893640 1 -- 172.21.15.46:0/2375783527 learned_addr learned my addr 172.21.15.46:0/2375783527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.712+0000 7f2972895640 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7088ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.713+0000 7f2972895640 1 -- 172.21.15.46:0/2375783527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 msgr2=0x5596d7093770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.713+0000 7f2972895640 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d7093770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.713+0000 7f2972895640 1 -- 172.21.15.46:0/2375783527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 msgr2=0x5596d70960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.713+0000 7f2972895640 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.713+0000 7f2972895640 1 -- 172.21.15.46:0/2375783527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d6f94340 con 0x5596d7088af0 2024-09-12T03:03:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.714+0000 7f2972895640 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7088ed0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2964006010 tx=0x7f2964030940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dd025c521eea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.719+0000 7f29637fe640 1 -- 172.21.15.46:0/1874788517 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2968018020 con 0x7f296c1009d0 2024-09-12T03:03:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.720+0000 7f29637fe640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 0x7f29580a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.720+0000 7f29637fe640 1 -- 172.21.15.46:0/1874788517 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f29680cebc0 con 0x7f296c1009d0 2024-09-12T03:03:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.715+0000 7f2960bfe640 1 -- 172.21.15.46:0/2375783527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964042030 con 0x5596d7088af0 2024-09-12T03:03:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.721+0000 7f2960bfe640 1 -- 172.21.15.46:0/2375783527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2964037040 con 0x5596d7088af0 2024-09-12T03:03:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.721+0000 7f2960bfe640 1 -- 172.21.15.46:0/2375783527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964036070 con 0x5596d7088af0 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.722+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 0x7f29580a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.722+0000 7f2977095740 1 -- 172.21.15.46:0/2375783527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 msgr2=0x5596d7088ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.722+0000 7f2977095740 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7088ed0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2964006010 tx=0x7f2964030940 comp rx=0 tx=0).stop 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.723+0000 7f2977095740 1 -- 172.21.15.46:0/2375783527 shutdown_connections 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.723+0000 7f2977095740 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.723+0000 7f2977095740 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d7093770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.723+0000 7f2977095740 1 --2- 172.21.15.46:0/2375783527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7088ed0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.723+0000 7f2977095740 1 -- 172.21.15.46:0/2375783527 >> 172.21.15.46:0/2375783527 conn(0x5596d7085060 msgr2=0x5596d7085440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.731+0000 7f2972094640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 0x7f29580a5480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5596d7085ec0 tx=0x7f2968004750 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.739+0000 7f2977095740 1 -- 172.21.15.46:0/2375783527 shutdown_connections 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.746+0000 7f2977095740 1 -- 172.21.15.46:0/2375783527 wait complete. 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.746+0000 7f2977095740 1 -- start start 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.746+0000 7f2977095740 1 Processor -- start 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7090b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d70910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596d7096880 con 0x5596d7088af0 2024-09-12T03:03:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5596d6f76290 con 0x5596d7093cb0 2024-09-12T03:03:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596d7096a00 con 0x5596d7089410 2024-09-12T03:03:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.747+0000 7f2977095740 10 client.0 _ll_register_callbacks cb 0x5596d6fa7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.749+0000 7f2972895640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.749+0000 7f2972895640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:52384/0 (socket says 172.21.15.46:52384) 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.749+0000 7f2972895640 1 -- 172.21.15.46:0/204548190 learned_addr learned my addr 172.21.15.46:0/204548190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 -- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 msgr2=0x5596d70910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d70910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 -- 172.21.15.46:0/204548190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 msgr2=0x5596d7090b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7090b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 -- 172.21.15.46:0/204548190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d6d00210 con 0x5596d7093cb0 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2972895640 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296402ac80 tx=0x7f29640098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ddc2b01abc12e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.750+0000 7f2955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.751+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964050060 con 0x5596d7093cb0 2024-09-12T03:03:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.751+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f29640305a0 con 0x5596d7093cb0 2024-09-12T03:03:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.754+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2964037040 con 0x5596d7093cb0 2024-09-12T03:03:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.768+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2964036070 con 0x5596d7093cb0 2024-09-12T03:03:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.768+0000 7f2977095740 10 client.25934 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.768+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596d6dcc620 con 0x5596d7093cb0 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2964037040 con 0x5596d7093cb0 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f296403c040 con 0x5596d7093cb0 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 10 client.25934 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 20 client.25934 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d70a3e70 0x5596d70a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 4 client.25934 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.772+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d6ffaa20 con 0x5596d70a3e70 2024-09-12T03:03:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.774+0000 7f2971893640 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d70a3e70 0x5596d70a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.776+0000 7f2971893640 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d70a3e70 0x5596d70a62d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4c8e43d0efb260 server_cookie=91f207160998af9b in_seq=0 out_seq=0 2024-09-12T03:03:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.777+0000 7f2955ffb640 10 client.25934 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.781+0000 7f2955ffb640 1 -- 172.21.15.46:0/204548190 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596d6ffaa20 con 0x5596d70a3e70 2024-09-12T03:03:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.781+0000 7f2955ffb640 10 client.25934 handle_command_reply: tid=0 2024-09-12T03:03:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.782+0000 7f2977095740 1 client.25934 shutdown 2024-09-12T03:03:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.792+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 msgr2=0x5596d7091610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.792+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296402ac80 tx=0x7f29640098d0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 shutdown_connections 2024-09-12T03:03:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d70a3e70 0x5596d70a62d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d7091610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7089410 0x5596d70910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 --2- 172.21.15.46:0/204548190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7090b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 >> 172.21.15.46:0/204548190 conn(0x5596d7085060 msgr2=0x5596d708ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.794+0000 7f2955ffb640 0 client.25934 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:03:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.801+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 shutdown_connections 2024-09-12T03:03:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.802+0000 7f2977095740 1 -- 172.21.15.46:0/204548190 wait complete. 2024-09-12T03:03:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.802+0000 7f2977095740 20 client.25934 trim_cache size 0 max 16384 2024-09-12T03:03:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.873+0000 7f2977095740 1 Processor -- start 2024-09-12T03:03:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.873+0000 7f2977095740 1 -- start start 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.873+0000 7f2977095740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7085440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.874+0000 7f2977095740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 0x5596d7091440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.874+0000 7f2977095740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7093cb0 0x5596d70abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.874+0000 7f2977095740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29500129c0 con 0x5596d7088af0 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.874+0000 7f2977095740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2950010230 con 0x5596d7089410 2024-09-12T03:03:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.874+0000 7f2977095740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f295000e080 con 0x5596d7093cb0 2024-09-12T03:03:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.876+0000 7f2972895640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7093cb0 0x5596d70abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.876+0000 7f2972895640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7085440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.876+0000 7f2972895640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7085440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39602/0 (socket says 172.21.15.46:39602) 2024-09-12T03:03:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.876+0000 7f2972895640 1 -- 172.21.15.46:0/3072805069 learned_addr learned my addr 172.21.15.46:0/3072805069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.877+0000 7f2971893640 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 0x5596d7091440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.878+0000 7f2971893640 1 -- 172.21.15.46:0/3072805069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7093cb0 msgr2=0x5596d70abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.878+0000 7f2971893640 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7093cb0 0x5596d70abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.878+0000 7f2971893640 1 -- 172.21.15.46:0/3072805069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 msgr2=0x5596d7085440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.878+0000 7f2971893640 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7085440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.878+0000 7f2971893640 1 -- 172.21.15.46:0/3072805069 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d708fb20 con 0x5596d7089410 2024-09-12T03:03:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.882+0000 7f2971893640 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 0x5596d7091440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5596d7092140 tx=0x7f295c004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9d898638eb095d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.882+0000 7f2957fff640 1 -- 172.21.15.46:0/3072805069 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c043020 con 0x5596d7089410 2024-09-12T03:03:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.883+0000 7f2957fff640 1 -- 172.21.15.46:0/3072805069 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f295c003ca0 con 0x5596d7089410 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 -- 172.21.15.46:0/3072805069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 msgr2=0x5596d7091440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 0x5596d7091440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5596d7092140 tx=0x7f295c004410 comp rx=0 tx=0).stop 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 -- 172.21.15.46:0/3072805069 shutdown_connections 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7093cb0 0x5596d70abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7089410 0x5596d7091440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 --2- 172.21.15.46:0/3072805069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7088af0 0x5596d7085440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 -- 172.21.15.46:0/3072805069 >> 172.21.15.46:0/3072805069 conn(0x5596d7085060 msgr2=0x5596d70841c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 -- 172.21.15.46:0/3072805069 shutdown_connections 2024-09-12T03:03:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.884+0000 7f2977095740 1 -- 172.21.15.46:0/3072805069 wait complete. 2024-09-12T03:03:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 -- start start 2024-09-12T03:03:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 Processor -- start 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7106ba0 0x5596d7107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f295000e6f0 con 0x5596d7106ba0 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f295000fb60 con 0x5596d7093cb0 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f295000edc0 con 0x5596d7088af0 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.885+0000 7f2977095740 10 client.0 _ll_register_callbacks cb 0x7f295c0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.886+0000 7f2972895640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.886+0000 7f2972895640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:59812/0 (socket says 172.21.15.46:59812) 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.886+0000 7f2972895640 1 -- 172.21.15.46:0/390530680 learned_addr learned my addr 172.21.15.46:0/390530680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.886+0000 7f2972895640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7106ba0 0x5596d7107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.886+0000 7f2971893640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2972895640 1 -- 172.21.15.46:0/390530680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 msgr2=0x5596d70979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2972895640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2972895640 1 -- 172.21.15.46:0/390530680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7106ba0 msgr2=0x5596d7107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2972895640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7106ba0 0x5596d7107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2972895640 1 -- 172.21.15.46:0/390530680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d7089240 con 0x5596d7088af0 2024-09-12T03:03:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.887+0000 7f2971893640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.888+0000 7f2972895640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2964034040 tx=0x7f296402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7355b95a34cb26b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.888+0000 7f2955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:03:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.888+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964064060 con 0x5596d7088af0 2024-09-12T03:03:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.888+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596d6ff4d70 con 0x5596d7088af0 2024-09-12T03:03:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.888+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f2964031640 con 0x5596d7088af0 2024-09-12T03:03:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.891+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2964050040 con 0x5596d7088af0 2024-09-12T03:03:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.892+0000 7f2977095740 10 client.25879 fetch_fsmap learned FSMap version 39 2024-09-12T03:03:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.892+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596d710e7f0 con 0x5596d7088af0 2024-09-12T03:03:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.896+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5596d6ff4d70 con 0x5596d7088af0 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f296403e5b0 con 0x5596d7088af0 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 10 client.25879 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 20 client.25879 populate_metadata read hostname 'smithi046' 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d710eef0 0x5596d7111310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 4 client.25879 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:03:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.898+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596d6f71900 con 0x5596d710eef0 2024-09-12T03:03:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.905+0000 7f2971893640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d710eef0 0x5596d7111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.913+0000 7f2971893640 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d710eef0 0x5596d7111310 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f4e9094a0cfd90 server_cookie=646e9909204986dd in_seq=0 out_seq=0 2024-09-12T03:03:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:03:59.913+0000 7f2955ffb640 10 client.25879 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.151+0000 7f2955ffb640 1 -- 172.21.15.46:0/390530680 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596d6f71900 con 0x5596d710eef0 2024-09-12T03:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.151+0000 7f2955ffb640 10 client.25879 handle_command_reply: tid=0 2024-09-12T03:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.151+0000 7f2977095740 1 client.25879 shutdown 2024-09-12T03:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.151+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 msgr2=0x5596d7097490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.151+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2964034040 tx=0x7f296402a710 comp rx=0 tx=0).stop 2024-09-12T03:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 shutdown_connections 2024-09-12T03:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5596d710eef0 0x5596d7111310 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596d7106ba0 0x5596d7107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5596d7093cb0 0x5596d70979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 --2- 172.21.15.46:0/390530680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596d7088af0 0x5596d7097490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 >> 172.21.15.46:0/390530680 conn(0x5596d7085060 msgr2=0x5596d70923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2955ffb640 0 client.25879 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 shutdown_connections 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 1 -- 172.21.15.46:0/390530680 wait complete. 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.152+0000 7f2977095740 20 client.25879 trim_cache size 0 max 16384 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:04:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:04:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.155+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 msgr2=0x7f29580a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.155+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 0x7f29580a5480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5596d7085ec0 tx=0x7f2968004750 comp rx=0 tx=0).stop 2024-09-12T03:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.155+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 msgr2=0x7f296c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.155+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 0x7f296c19acf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fd0 tx=0x7f296800b7b0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 shutdown_connections 2024-09-12T03:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f29580a2fc0 0x7f29580a5480 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f296c1069e0 0x7f296c1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101370 0x7f296c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 --2- 172.21.15.46:0/1874788517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f296c1009d0 0x7f296c19acf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.160+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 >> 172.21.15.46:0/1874788517 conn(0x7f296c0fc6b0 msgr2=0x7f296c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.166+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 shutdown_connections 2024-09-12T03:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:00.166+0000 7f29742f8640 1 -- 172.21.15.46:0/1874788517 wait complete. 2024-09-12T03:04:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:00 smithi145 ceph-mon[34674]: pgmap v434: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 3.90k op/s 2024-09-12T03:04:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:00 smithi196 ceph-mon[34587]: pgmap v434: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 3.90k op/s 2024-09-12T03:04:00.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:00 smithi046 ceph-mon[30951]: pgmap v434: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 3.90k op/s 2024-09-12T03:04:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:02 smithi196 ceph-mon[34587]: pgmap v435: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-12T03:04:02.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:02 smithi046 ceph-mon[30951]: pgmap v435: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-12T03:04:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:02 smithi145 ceph-mon[34674]: pgmap v435: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-12T03:04:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:04 smithi196 ceph-mon[34587]: pgmap v436: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-12T03:04:04.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:04 smithi046 ceph-mon[30951]: pgmap v436: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-12T03:04:04.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:04 smithi145 ceph-mon[34674]: pgmap v436: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2024-09-12T03:04:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:06 smithi196 ceph-mon[34587]: pgmap v437: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:06.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:06 smithi046 ceph-mon[30951]: pgmap v437: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:06.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:06 smithi145 ceph-mon[34674]: pgmap v437: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:08 smithi196 ceph-mon[34587]: pgmap v438: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-09-12T03:04:08.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:08 smithi046 ceph-mon[30951]: pgmap v438: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-09-12T03:04:08.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:08 smithi145 ceph-mon[34674]: pgmap v438: 129 pgs: 5 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 120 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-09-12T03:04:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:10 smithi196 ceph-mon[34587]: pgmap v439: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 3.67k op/s 2024-09-12T03:04:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:10 smithi046 ceph-mon[30951]: pgmap v439: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 3.67k op/s 2024-09-12T03:04:10.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:10 smithi145 ceph-mon[34674]: pgmap v439: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 3.67k op/s 2024-09-12T03:04:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:12 smithi196 ceph-mon[34587]: pgmap v440: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-12T03:04:12.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:12 smithi046 ceph-mon[30951]: pgmap v440: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-12T03:04:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:12 smithi145 ceph-mon[34674]: pgmap v440: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-09-12T03:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.384+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2708079143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 msgr2=0x7f2a3c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.384+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2708079143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c109a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009900 tx=0x7f2a2402a410 comp rx=0 tx=0).stop 2024-09-12T03:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.386+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2708079143 shutdown_connections 2024-09-12T03:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.386+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2708079143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c11b140 0x7f2a3c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.386+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2708079143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c109a30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.386+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2708079143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.386+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2708079143 >> 172.21.15.46:0/2708079143 conn(0x7f2a3c06cbf0 msgr2=0x7f2a3c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.388+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2708079143 shutdown_connections 2024-09-12T03:04:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.388+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2708079143 wait complete. 2024-09-12T03:04:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 Processor -- start 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 -- start start 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c1117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c11b140 0x7f2a3c111d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1209c0 con 0x7f2a3c11b140 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c120840 con 0x7f2a3c1095d0 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.389+0000 7f2a3bfff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c120b40 con 0x7f2a3c1091f0 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.390+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.390+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c118750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:36938/0 (socket says 172.21.15.46:36938) 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.390+0000 7f2a3affd640 1 -- 172.21.15.46:0/2907141334 learned_addr learned my addr 172.21.15.46:0/2907141334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.390+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c1117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.393+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/2907141334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 msgr2=0x7f2a3c118750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.393+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c118750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.393+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c11b140 msgr2=0x7f2a3c111d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.393+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c11b140 0x7f2a3c111d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.393+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/2907141334 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c1123b0 con 0x7f2a3c1095d0 2024-09-12T03:04:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.394+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c1117d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009900 tx=0x7f2a24005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.395+0000 7f2a40835640 1 -- 172.21.15.46:0/2907141334 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2403f070 con 0x7f2a3c1095d0 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.395+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c116270 con 0x7f2a3c1095d0 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.395+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c116760 con 0x7f2a3c1095d0 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.398+0000 7f2a445ce740 1 Processor -- start 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 -- start start 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a40835640 1 -- 172.21.15.46:0/2907141334 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a24004460 con 0x7f2a3c1095d0 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a40835640 1 -- 172.21.15.46:0/2907141334 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24035030 con 0x7f2a3c1095d0 2024-09-12T03:04:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 0x558957e200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e2ae80 0x558957e2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558957d842f0 con 0x558957e1fcc0 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558957d2c270 con 0x558957e2ae80 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a445ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558957d802b0 con 0x558957e205e0 2024-09-12T03:04:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 0x558957e200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e2a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 0x558957e200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38340/0 (socket says 172.21.15.46:38340) 2024-09-12T03:04:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.399+0000 7f2a3b7fe640 1 -- 172.21.15.46:0/1647598362 learned_addr learned my addr 172.21.15.46:0/1647598362 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.400+0000 7f2a3affd640 1 -- 172.21.15.46:0/1647598362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e2ae80 msgr2=0x558957e2d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.400+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e2ae80 0x558957e2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.400+0000 7f2a3affd640 1 -- 172.21.15.46:0/1647598362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 msgr2=0x558957e200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.400+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 0x558957e200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.400+0000 7f2a3affd640 1 -- 172.21.15.46:0/1647598362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558957d2b340 con 0x558957e205e0 2024-09-12T03:04:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.404+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e2a940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a340058c0 tx=0x7f2a340107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5152c8a2e14460 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.406+0000 7f2a40835640 1 -- 172.21.15.46:0/2907141334 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a24042070 con 0x7f2a3c1095d0 2024-09-12T03:04:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.407+0000 7f2a31ffb640 1 -- 172.21.15.46:0/1647598362 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004230 con 0x558957e205e0 2024-09-12T03:04:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.408+0000 7f2a31ffb640 1 -- 172.21.15.46:0/1647598362 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3400be90 con 0x558957e205e0 2024-09-12T03:04:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.408+0000 7f2a31ffb640 1 -- 172.21.15.46:0/1647598362 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a340116c0 con 0x558957e205e0 2024-09-12T03:04:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.409+0000 7f2a445ce740 1 -- 172.21.15.46:0/1647598362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 msgr2=0x558957e2a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.409+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e2a940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a340058c0 tx=0x7f2a340107a0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.412+0000 7f2a40835640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 0x7f2a280a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.412+0000 7f2a40835640 1 -- 172.21.15.46:0/2907141334 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f2a240efd60 con 0x7f2a3c1095d0 2024-09-12T03:04:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.413+0000 7f2a445ce740 1 -- 172.21.15.46:0/1647598362 shutdown_connections 2024-09-12T03:04:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.413+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e2ae80 0x558957e2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.413+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e2a940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.413+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1647598362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e1fcc0 0x558957e200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.413+0000 7f2a445ce740 1 -- 172.21.15.46:0/1647598362 >> 172.21.15.46:0/1647598362 conn(0x558957e1c230 msgr2=0x558957e1c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.415+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 0x7f2a280a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.415+0000 7f2a445ce740 1 -- 172.21.15.46:0/1647598362 shutdown_connections 2024-09-12T03:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.417+0000 7f2a445ce740 1 -- 172.21.15.46:0/1647598362 wait complete. 2024-09-12T03:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.417+0000 7f2a445ce740 1 -- start start 2024-09-12T03:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 Processor -- start 2024-09-12T03:04:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e1fcc0 0x558957e27d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558957e2da50 con 0x558957e2ae80 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558957d0d290 con 0x558957e205e0 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558957e2dbd0 con 0x558957e1fcc0 2024-09-12T03:04:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.418+0000 7f2a445ce740 10 client.0 _ll_register_callbacks cb 0x558957d3e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:45638/0 (socket says 172.21.15.46:45638) 2024-09-12T03:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/1408337699 learned_addr learned my addr 172.21.15.46:0/1408337699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e1fcc0 0x558957e27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e1fcc0 msgr2=0x558957e27d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e1fcc0 0x558957e27d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/1408337699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 msgr2=0x558957e287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.420+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/1408337699 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558957a97210 con 0x558957e205e0 2024-09-12T03:04:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.421+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 0x7f2a280a53c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0092f0 tx=0x7f2a2c0094e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.428+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a240c4620 tx=0x7f2a240bbae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f6bdf66a89c8df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.428+0000 7f2a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24049040 con 0x558957e205e0 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3400be90 con 0x558957e205e0 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f2a240315b0 con 0x558957e205e0 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a240bc5e0 con 0x558957e205e0 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a445ce740 10 client.25949 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558957b63620 con 0x558957e205e0 2024-09-12T03:04:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.429+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3400be90 con 0x558957e205e0 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2a24003ae0 con 0x558957e205e0 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 10 client.25949 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 20 client.25949 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957e3b0e0 0x558957e3d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 4 client.25949 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558957d91a20 con 0x558957e3b0e0 2024-09-12T03:04:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957e3b0e0 0x558957e3d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a3affd640 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957e3b0e0 0x558957e3d500 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d32e90d3dfa087a server_cookie=73af0543004a8d46 in_seq=0 out_seq=0 2024-09-12T03:04:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.430+0000 7f2a1affd640 10 client.25949 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.431+0000 7f2a1affd640 1 -- 172.21.15.46:0/1408337699 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558957d91a20 con 0x558957e3b0e0 2024-09-12T03:04:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.431+0000 7f2a1affd640 10 client.25949 handle_command_reply: tid=0 2024-09-12T03:04:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.431+0000 7f2a445ce740 1 client.25949 shutdown 2024-09-12T03:04:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.431+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 msgr2=0x558957e28290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.431+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a240c4620 tx=0x7f2a240bbae0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 shutdown_connections 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957e3b0e0 0x558957e3d500 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e205e0 0x558957e28290 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 --2- 172.21.15.46:0/1408337699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e1fcc0 0x558957e27d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 >> 172.21.15.46:0/1408337699 conn(0x558957e1c230 msgr2=0x558957e21f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a1affd640 0 client.25949 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 shutdown_connections 2024-09-12T03:04:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 1 -- 172.21.15.46:0/1408337699 wait complete. 2024-09-12T03:04:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.432+0000 7f2a445ce740 20 client.25949 trim_cache size 0 max 16384 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 Processor -- start 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 -- start start 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e1bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c0128d0 con 0x558957e2ae80 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c010140 con 0x558957e1fcc0 2024-09-12T03:04:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a445ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00df90 con 0x558957e205e0 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38358/0 (socket says 172.21.15.46:38358) 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 -- 172.21.15.46:0/646598913 learned_addr learned my addr 172.21.15.46:0/646598913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3affd640 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e1bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 -- 172.21.15.46:0/646598913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 msgr2=0x558957e28600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e28600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 -- 172.21.15.46:0/646598913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 msgr2=0x558957e1bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e1bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.449+0000 7f2a3b7fe640 1 -- 172.21.15.46:0/646598913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558957e26ce0 con 0x558957e2ae80 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c012550 tx=0x7f2a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7994f2c4dd4ac0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a317fa640 1 -- 172.21.15.46:0/646598913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c023070 con 0x558957e2ae80 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a317fa640 1 -- 172.21.15.46:0/646598913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c01eaf0 con 0x558957e2ae80 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a317fa640 1 -- 172.21.15.46:0/646598913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c011b10 con 0x558957e2ae80 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a445ce740 1 -- 172.21.15.46:0/646598913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 msgr2=0x558957e1b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.450+0000 7f2a445ce740 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c012550 tx=0x7f2a2c004a40 comp rx=0 tx=0).stop 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 -- 172.21.15.46:0/646598913 shutdown_connections 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e2ae80 0x558957e1b150 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e205e0 0x558957e28600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 --2- 172.21.15.46:0/646598913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 -- 172.21.15.46:0/646598913 >> 172.21.15.46:0/646598913 conn(0x558957e1c230 msgr2=0x558957e1aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 -- 172.21.15.46:0/646598913 shutdown_connections 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 -- 172.21.15.46:0/646598913 wait complete. 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 -- start start 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.451+0000 7f2a445ce740 1 Processor -- start 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e33450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e2ae80 0x558957e2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00e600 con 0x558957e205e0 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00fa70 con 0x558957e1fcc0 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00ecd0 con 0x558957e2ae80 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38372/0 (socket says 172.21.15.46:38372) 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/451608618 learned_addr learned my addr 172.21.15.46:0/451608618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a445ce740 10 client.0 _ll_register_callbacks cb 0x7f2a24003fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3affd640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e33450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3b7fe640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e2ae80 0x558957e2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e2ae80 msgr2=0x558957e2ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e2ae80 0x558957e2ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/451608618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 msgr2=0x558957e33450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e33450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 -- 172.21.15.46:0/451608618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558957e2f2f0 con 0x558957e205e0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a3a7fc640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a24033000 tx=0x7f2a240056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9edf22481b8575 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.452+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24049040 con 0x558957e205e0 2024-09-12T03:04:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9479+0+0 (secure 0 0 0) 0x7f2a240c46c0 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2c01eaf0 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a240bc140 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 10 client.16347 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558957ea5c00 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c01eaf0 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2a240b7ba0 con 0x558957e205e0 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 10 client.16347 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 20 client.16347 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957ea6240 0x558957ea8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.453+0000 7f2a445ce740 4 client.16347 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:04:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.454+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558957d8bd70 con 0x558957ea6240 2024-09-12T03:04:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.454+0000 7f2a3affd640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957ea6240 0x558957ea8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.454+0000 7f2a1affd640 10 client.16347 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:13.454+0000 7f2a3affd640 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957ea6240 0x558957ea8660 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e72a21ff6199e22 server_cookie=92fdee4a6e59635b in_seq=0 out_seq=0 2024-09-12T03:04:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:14 smithi196 ceph-mon[34587]: pgmap v441: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:04:14.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:14 smithi046 ceph-mon[30951]: pgmap v441: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:04:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:14 smithi145 ceph-mon[34674]: pgmap v441: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-09-12T03:04:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.959+0000 7f2a1affd640 1 -- 172.21.15.46:0/451608618 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558957d8bd70 con 0x558957ea6240 2024-09-12T03:04:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.959+0000 7f2a1affd640 10 client.16347 handle_command_reply: tid=0 2024-09-12T03:04:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.959+0000 7f2a445ce740 1 client.16347 shutdown 2024-09-12T03:04:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.967+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 msgr2=0x558957e2e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.967+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a24033000 tx=0x7f2a240056e0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 shutdown_connections 2024-09-12T03:04:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x558957ea6240 0x558957ea8660 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558957e2ae80 0x558957e2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558957e205e0 0x558957e2e770 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 --2- 172.21.15.46:0/451608618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558957e1fcc0 0x558957e33450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 >> 172.21.15.46:0/451608618 conn(0x558957e1c230 msgr2=0x558957e1b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a1affd640 0 client.16347 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 shutdown_connections 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 1 -- 172.21.15.46:0/451608618 wait complete. 2024-09-12T03:04:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.968+0000 7f2a445ce740 20 client.16347 trim_cache size 0 max 16384 2024-09-12T03:04:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:04:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:04:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:04:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 msgr2=0x7f2a280a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 0x7f2a280a53c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0092f0 tx=0x7f2a2c0094e0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 msgr2=0x7f2a3c1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c1117d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009900 tx=0x7f2a24005570 comp rx=0 tx=0).stop 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 shutdown_connections 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f2a280a2f00 0x7f2a280a53c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c11b140 0x7f2a3c111d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a3c1095d0 0x7f2a3c1117d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 --2- 172.21.15.46:0/2907141334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c1091f0 0x7f2a3c118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.980+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 >> 172.21.15.46:0/2907141334 conn(0x7f2a3c06cbf0 msgr2=0x7f2a3c11ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.985+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 shutdown_connections 2024-09-12T03:04:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:14.985+0000 7f2a3bfff640 1 -- 172.21.15.46:0/2907141334 wait complete. 2024-09-12T03:04:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:16 smithi196 ceph-mon[34587]: pgmap v442: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:16.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:16 smithi046 ceph-mon[30951]: pgmap v442: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:16.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:16 smithi145 ceph-mon[34674]: pgmap v442: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:17.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:17 smithi046 ceph-mon[30951]: pgmap v443: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-12T03:04:17.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:17 smithi046 ceph-mon[30951]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:04:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:17 smithi145 ceph-mon[34674]: pgmap v443: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-12T03:04:17.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:17 smithi145 ceph-mon[34674]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:04:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:17 smithi196 ceph-mon[34587]: pgmap v443: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-12T03:04:17.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:17 smithi196 ceph-mon[34587]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:04:20.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:19 smithi145 ceph-mon[34674]: pgmap v445: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:04:20.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:19 smithi196 ceph-mon[34587]: pgmap v445: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:04:20.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:19 smithi046 ceph-mon[30951]: pgmap v445: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.17k op/s 2024-09-12T03:04:22.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:21 smithi145 ceph-mon[34674]: pgmap v446: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:04:22.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:21 smithi196 ceph-mon[34587]: pgmap v446: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:04:22.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:21 smithi046 ceph-mon[30951]: pgmap v446: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-12T03:04:24.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:23 smithi145 ceph-mon[34674]: pgmap v447: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:04:24.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:23 smithi196 ceph-mon[34587]: pgmap v447: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:04:24.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:23 smithi046 ceph-mon[30951]: pgmap v447: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 2.84k op/s 2024-09-12T03:04:26.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:25 smithi145 ceph-mon[34674]: pgmap v448: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-12T03:04:26.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:25 smithi196 ceph-mon[34587]: pgmap v448: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-12T03:04:26.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:25 smithi046 ceph-mon[30951]: pgmap v448: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 16 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-12T03:04:28.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:27 smithi145 ceph-mon[34674]: pgmap v449: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:28.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:27 smithi196 ceph-mon[34587]: pgmap v449: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.270+0000 7fd9aac4c640 1 -- 172.21.15.46:0/943763594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41095d0 msgr2=0x7fd9a4109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.270+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/943763594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41095d0 0x7fd9a4109a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009ca0 tx=0x7fd99c02a8d0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.281+0000 7fd9aac4c640 1 -- 172.21.15.46:0/943763594 shutdown_connections 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.281+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/943763594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 0x7fd9a411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.281+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/943763594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41095d0 0x7fd9a4109a10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.281+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/943763594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41091f0 0x7fd9a4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.281+0000 7fd9aac4c640 1 -- 172.21.15.46:0/943763594 >> 172.21.15.46:0/943763594 conn(0x7fd9a406cc60 msgr2=0x7fd9a406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.282+0000 7fd9aac4c640 1 -- 172.21.15.46:0/943763594 shutdown_connections 2024-09-12T03:04:28.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:27 smithi046 ceph-mon[30951]: pgmap v449: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 16 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.282+0000 7fd9aac4c640 1 -- 172.21.15.46:0/943763594 wait complete. 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 Processor -- start 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 -- start start 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 0x7fd9a41a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 0x7fd9a41a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 0x7fd9a41ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4120a10 con 0x7fd9a41095d0 2024-09-12T03:04:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4120890 con 0x7fd9a411b140 2024-09-12T03:04:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.283+0000 7fd9aac4c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4120b90 con 0x7fd9a41091f0 2024-09-12T03:04:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.285+0000 7fd9a3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 0x7fd9a41a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.285+0000 7fd9a3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 0x7fd9a41a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34050/0 (socket says 172.21.15.46:34050) 2024-09-12T03:04:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.285+0000 7fd9a3fff640 1 -- 172.21.15.46:0/2167029795 learned_addr learned my addr 172.21.15.46:0/2167029795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.288+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 0x7fd9a41a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.288+0000 7fd9a91e9640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 0x7fd9a41ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.291+0000 7fd9a89e8640 1 -- 172.21.15.46:0/2167029795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 msgr2=0x7fd9a41ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.291+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 0x7fd9a41ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.291+0000 7fd9a89e8640 1 -- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 msgr2=0x7fd9a41a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.291+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 0x7fd9a41a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.291+0000 7fd9a89e8640 1 -- 172.21.15.46:0/2167029795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41abef0 con 0x7fd9a41091f0 2024-09-12T03:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.298+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 0x7fd9a41a6f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d700 tx=0x7fd98c00dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.302+0000 7fd9a1ffb640 1 -- 172.21.15.46:0/2167029795 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c017070 con 0x7fd9a41091f0 2024-09-12T03:04:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.302+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41ac1e0 con 0x7fd9a41091f0 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.302+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a4110920 con 0x7fd9a41091f0 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 Processor -- start 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 -- start start 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266c9cc0 0x5623266ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56232662e2f0 con 0x5623266ca5e0 2024-09-12T03:04:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5623265d6270 con 0x5623266d4e80 2024-09-12T03:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.306+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56232662a2b0 con 0x5623266c9cc0 2024-09-12T03:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a1ffb640 1 -- 172.21.15.46:0/2167029795 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98c009820 con 0x7fd9a41091f0 2024-09-12T03:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34054/0 (socket says 172.21.15.46:34054) 2024-09-12T03:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1349310982 learned_addr learned my addr 172.21.15.46:0/1349310982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a91e9640 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266c9cc0 0x5623266ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.307+0000 7fd9a1ffb640 1 -- 172.21.15.46:0/2167029795 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c004c40 con 0x7fd9a41091f0 2024-09-12T03:04:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.310+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1349310982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266c9cc0 msgr2=0x5623266ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.310+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266c9cc0 0x5623266ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.310+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1349310982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 msgr2=0x5623266d7270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:04:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.310+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266d7270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.310+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1349310982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623265d5340 con 0x5623266ca5e0 2024-09-12T03:04:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.312+0000 7fd9a1ffb640 1 -- 172.21.15.46:0/2167029795 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd98c048b40 con 0x7fd9a41091f0 2024-09-12T03:04:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.313+0000 7fd9a1ffb640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 0x7fd9900a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.314+0000 7fd9a1ffb640 1 -- 172.21.15.46:0/2167029795 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fd98c0d0380 con 0x7fd9a41091f0 2024-09-12T03:04:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.314+0000 7fd9a91e9640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 0x7fd9900a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.317+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d6d0 tx=0x7fd98c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d43b48f3b38f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.318+0000 7fd9a91e9640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 0x7fd9900a53c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd994005d80 tx=0x7fd99400b420 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.324+0000 7fd99affd640 1 -- 172.21.15.46:0/1349310982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c020070 con 0x5623266ca5e0 2024-09-12T03:04:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.324+0000 7fd99affd640 1 -- 172.21.15.46:0/1349310982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98c0d0160 con 0x5623266ca5e0 2024-09-12T03:04:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.325+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1349310982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 msgr2=0x5623266d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.325+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d6d0 tx=0x7fd98c017000 comp rx=0 tx=0).stop 2024-09-12T03:04:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.332+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1349310982 shutdown_connections 2024-09-12T03:04:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.332+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.332+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266ca5e0 0x5623266d4940 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.332+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1349310982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266c9cc0 0x5623266ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.332+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1349310982 >> 172.21.15.46:0/1349310982 conn(0x5623266c6230 msgr2=0x5623266c6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.337+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1349310982 shutdown_connections 2024-09-12T03:04:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.337+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1349310982 wait complete. 2024-09-12T03:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.337+0000 7fd9ad9e7740 1 -- start start 2024-09-12T03:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 Processor -- start 2024-09-12T03:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266ca5e0 0x5623266d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266d4e80 0x5623266d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623266d7a50 con 0x5623266c9cc0 2024-09-12T03:04:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5623265b7290 con 0x5623266ca5e0 2024-09-12T03:04:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623266d7bd0 con 0x5623266d4e80 2024-09-12T03:04:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.338+0000 7fd9ad9e7740 10 client.0 _ll_register_callbacks cb 0x5623265e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.339+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.339+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34060/0 (socket says 172.21.15.46:34060) 2024-09-12T03:04:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.339+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1062915898 learned_addr learned my addr 172.21.15.46:0/1062915898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266d4e80 msgr2=0x5623266d27b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:04:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266d4e80 0x5623266d27b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1062915898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266ca5e0 msgr2=0x5623266d2270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:04:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266ca5e0 0x5623266d2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1062915898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562326341210 con 0x5623266c9cc0 2024-09-12T03:04:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.340+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0cd7f0 tx=0x7fd98c098fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce35400edb6a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.342+0000 7fd983fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.342+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c020040 con 0x5623266c9cc0 2024-09-12T03:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.342+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fd98c0cd930 con 0x5623266c9cc0 2024-09-12T03:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.346+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56232663ba20 con 0x5623266c9cc0 2024-09-12T03:04:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.356+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd98c09c700 con 0x5623266c9cc0 2024-09-12T03:04:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.358+0000 7fd9ad9e7740 10 client.16359 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.358+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56232640d620 con 0x5623266c9cc0 2024-09-12T03:04:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.361+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56232663ba20 con 0x5623266c9cc0 2024-09-12T03:04:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.361+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd98c0bc020 con 0x5623266c9cc0 2024-09-12T03:04:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 10 client.16359 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 20 client.16359 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5623266e4c80 0x5623266e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 4 client.16359 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.364+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562326635d70 con 0x5623266e4c80 2024-09-12T03:04:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.366+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5623266e4c80 0x5623266e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.373+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5623266e4c80 0x5623266e7070 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d452efb3d3e638 server_cookie=7e2ec85eea105223 in_seq=0 out_seq=0 2024-09-12T03:04:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.375+0000 7fd983fff640 10 client.16359 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.376+0000 7fd983fff640 1 -- 172.21.15.46:0/1062915898 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562326635d70 con 0x5623266e4c80 2024-09-12T03:04:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.376+0000 7fd983fff640 10 client.16359 handle_command_reply: tid=0 2024-09-12T03:04:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.381+0000 7fd9ad9e7740 1 client.16359 shutdown 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 msgr2=0x5623266d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0cd7f0 tx=0x7fd98c098fa0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 shutdown_connections 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5623266e4c80 0x5623266e7070 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266d4e80 0x5623266d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266ca5e0 0x5623266d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1062915898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d1d30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.386+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 >> 172.21.15.46:0/1062915898 conn(0x5623266c6230 msgr2=0x5623266cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.387+0000 7fd983fff640 0 client.16359 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.394+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 shutdown_connections 2024-09-12T03:04:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.395+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1062915898 wait complete. 2024-09-12T03:04:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.398+0000 7fd9ad9e7740 20 client.16359 trim_cache size 0 max 16384 2024-09-12T03:04:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.449+0000 7fd9ad9e7740 1 Processor -- start 2024-09-12T03:04:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.449+0000 7fd9ad9e7740 1 -- start start 2024-09-12T03:04:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.450+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.453+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.453+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.453+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9840129f0 con 0x5623266c9cc0 2024-09-12T03:04:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.453+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd984010260 con 0x5623266d4e80 2024-09-12T03:04:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.453+0000 7fd9ad9e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd98400e0d0 con 0x5623266ca5e0 2024-09-12T03:04:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34068/0 (socket says 172.21.15.46:34068) 2024-09-12T03:04:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1614142965 learned_addr learned my addr 172.21.15.46:0/1614142965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a91e9640 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1614142965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 msgr2=0x5623266cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1614142965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 msgr2=0x5623266cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.455+0000 7fd9a89e8640 1 -- 172.21.15.46:0/1614142965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623266d7b50 con 0x5623266c9cc0 2024-09-12T03:04:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.456+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0990d0 tx=0x7fd98c09c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad735b6c2b799b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.457+0000 7fd99a7fc640 1 -- 172.21.15.46:0/1614142965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c020050 con 0x5623266c9cc0 2024-09-12T03:04:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.457+0000 7fd99a7fc640 1 -- 172.21.15.46:0/1614142965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98c0a50b0 con 0x5623266c9cc0 2024-09-12T03:04:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.459+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1614142965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 msgr2=0x5623266d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.459+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0990d0 tx=0x7fd98c09c4d0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1614142965 shutdown_connections 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266d4e80 0x5623266cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/1614142965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266c9cc0 0x5623266d25e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1614142965 >> 172.21.15.46:0/1614142965 conn(0x5623266c6230 msgr2=0x5623266c6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.463+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1614142965 shutdown_connections 2024-09-12T03:04:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.467+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/1614142965 wait complete. 2024-09-12T03:04:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.467+0000 7fd9ad9e7740 1 -- start start 2024-09-12T03:04:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.467+0000 7fd9ad9e7740 1 Processor -- start 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266d4e80 0x56232674d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9a89e8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:42512/0 (socket says 172.21.15.46:42512) 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9a89e8640 1 -- 172.21.15.46:0/676032271 learned_addr learned my addr 172.21.15.46:0/676032271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd98400e720 con 0x5623266d4e80 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd98400fb90 con 0x5623266c9cc0 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd98400edf0 con 0x5623266ca5e0 2024-09-12T03:04:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.468+0000 7fd9ad9e7740 10 client.0 _ll_register_callbacks cb 0x7fd98c0982d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.469+0000 7fd9a89e8640 1 -- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 msgr2=0x5623266e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.469+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.469+0000 7fd9a89e8640 1 -- 172.21.15.46:0/676032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266d4e80 msgr2=0x56232674d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.469+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266d4e80 0x56232674d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.469+0000 7fd9a89e8640 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232674daf0 con 0x5623266c9cc0 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.471+0000 7fd9a89e8640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd98c098b60 tx=0x7fd98c00ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0dea14f8cfc0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.471+0000 7fd983fff640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:04:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.473+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c051030 con 0x5623266c9cc0 2024-09-12T03:04:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.474+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c0a50b0 con 0x5623266c9cc0 2024-09-12T03:04:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.474+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fd98c0bc570 con 0x5623266c9cc0 2024-09-12T03:04:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.475+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd98c08e050 con 0x5623266c9cc0 2024-09-12T03:04:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.479+0000 7fd9ad9e7740 10 client.25967 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.479+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56232674daf0 con 0x5623266c9cc0 2024-09-12T03:04:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.480+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98c0a50b0 con 0x5623266c9cc0 2024-09-12T03:04:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.480+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd98c0bc040 con 0x5623266c9cc0 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 10 client.25967 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 20 client.25967 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56232674ef50 0x562326751390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 4 client.25967 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:04:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.484+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623265b2900 con 0x56232674ef50 2024-09-12T03:04:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.486+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56232674ef50 0x562326751390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.493+0000 7fd9a3fff640 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56232674ef50 0x562326751390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1039376077e0b5b0 server_cookie=5154043400712d0d in_seq=0 out_seq=0 2024-09-12T03:04:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:28.493+0000 7fd983fff640 10 client.25967 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:30.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:29 smithi145 ceph-mon[34674]: pgmap v450: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-12T03:04:30.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:29 smithi196 ceph-mon[34587]: pgmap v450: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-12T03:04:30.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:29 smithi046 ceph-mon[30951]: pgmap v450: 129 pgs: 16 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-09-12T03:04:32.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:31 smithi145 ceph-mon[34674]: pgmap v451: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-12T03:04:32.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:31 smithi196 ceph-mon[34587]: pgmap v451: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-12T03:04:32.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:31 smithi046 ceph-mon[30951]: pgmap v451: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-12T03:04:34.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:33 smithi145 ceph-mon[34674]: pgmap v452: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-12T03:04:34.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:33 smithi196 ceph-mon[34587]: pgmap v452: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-12T03:04:34.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:33 smithi046 ceph-mon[30951]: pgmap v452: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-12T03:04:36.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:35 smithi145 ceph-mon[34674]: pgmap v453: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:36.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:35 smithi196 ceph-mon[34587]: pgmap v453: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:36.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:35 smithi046 ceph-mon[30951]: pgmap v453: 129 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 84 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-12T03:04:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.724+0000 7fd983fff640 1 -- 172.21.15.46:0/676032271 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623265b2900 con 0x56232674ef50 2024-09-12T03:04:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.724+0000 7fd983fff640 10 client.25967 handle_command_reply: tid=0 2024-09-12T03:04:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.724+0000 7fd9ad9e7740 1 client.25967 shutdown 2024-09-12T03:04:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.724+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 msgr2=0x5623266e72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.724+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd98c098b60 tx=0x7fd98c00ba40 comp rx=0 tx=0).stop 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 shutdown_connections 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56232674ef50 0x562326751390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623266d4e80 0x56232674d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623266ca5e0 0x5623266e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 --2- 172.21.15.46:0/676032271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623266c9cc0 0x5623266e72a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 >> 172.21.15.46:0/676032271 conn(0x5623266c6230 msgr2=0x5623266c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd983fff640 0 client.25967 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 shutdown_connections 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 1 -- 172.21.15.46:0/676032271 wait complete. 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.725+0000 7fd9ad9e7740 20 client.25967 trim_cache size 0 max 16384 2024-09-12T03:04:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:04:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:04:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:04:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 msgr2=0x7fd9900a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 0x7fd9900a53c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd994005d80 tx=0x7fd99400b420 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 msgr2=0x7fd9a41a6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 0x7fd9a41a6f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d700 tx=0x7fd98c00dbc0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 shutdown_connections 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fd9900a2f00 0x7fd9900a53c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a411b140 0x7fd9a41ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a41095d0 0x7fd9a41a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 --2- 172.21.15.46:0/2167029795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a41091f0 0x7fd9a41a6f30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 >> 172.21.15.46:0/2167029795 conn(0x7fd9a406cc60 msgr2=0x7fd9a411b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 shutdown_connections 2024-09-12T03:04:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:37.726+0000 7fd9aac4c640 1 -- 172.21.15.46:0/2167029795 wait complete. 2024-09-12T03:04:38.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:37 smithi145 ceph-mon[34674]: pgmap v454: 129 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-12T03:04:38.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:37 smithi196 ceph-mon[34587]: pgmap v454: 129 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-12T03:04:38.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:37 smithi046 ceph-mon[30951]: pgmap v454: 129 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 86 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-12T03:04:40.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:39 smithi145 ceph-mon[34674]: pgmap v455: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-12T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:39 smithi196 ceph-mon[34587]: pgmap v455: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-12T03:04:40.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:39 smithi046 ceph-mon[30951]: pgmap v455: 129 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-12T03:04:42.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:41 smithi196 ceph-mon[34587]: pgmap v456: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-12T03:04:42.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:41 smithi046 ceph-mon[30951]: pgmap v456: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-12T03:04:42.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:41 smithi145 ceph-mon[34674]: pgmap v456: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-12T03:04:44.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:43 smithi196 ceph-mon[34587]: pgmap v457: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:04:44.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:43 smithi046 ceph-mon[30951]: pgmap v457: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:04:44.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:43 smithi145 ceph-mon[34674]: pgmap v457: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 3.40k op/s 2024-09-12T03:04:46.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:45 smithi196 ceph-mon[34587]: pgmap v458: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-09-12T03:04:46.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:45 smithi046 ceph-mon[30951]: pgmap v458: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-09-12T03:04:46.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:45 smithi145 ceph-mon[34674]: pgmap v458: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-09-12T03:04:48.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:47 smithi196 ceph-mon[34587]: pgmap v459: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-12T03:04:48.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:47 smithi046 ceph-mon[30951]: pgmap v459: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-12T03:04:48.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:47 smithi145 ceph-mon[34674]: pgmap v459: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-09-12T03:04:50.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:49 smithi196 ceph-mon[34587]: pgmap v460: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-12T03:04:50.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:49 smithi046 ceph-mon[30951]: pgmap v460: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-12T03:04:50.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:49 smithi145 ceph-mon[34674]: pgmap v460: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-09-12T03:04:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.866+0000 7fe065b66640 1 -- 172.21.15.46:0/1173323895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0601091f0 msgr2=0x7fe060109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.866+0000 7fe065b66640 1 --2- 172.21.15.46:0/1173323895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0601091f0 0x7fe060109630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe050009d10 tx=0x7fe05002b4e0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.868+0000 7fe065b66640 1 -- 172.21.15.46:0/1173323895 shutdown_connections 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.868+0000 7fe065b66640 1 --2- 172.21.15.46:0/1173323895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe06007b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.868+0000 7fe065b66640 1 --2- 172.21.15.46:0/1173323895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0601091f0 0x7fe060109630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.868+0000 7fe065b66640 1 --2- 172.21.15.46:0/1173323895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe06010d610 0x7fe06010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.868+0000 7fe065b66640 1 -- 172.21.15.46:0/1173323895 >> 172.21.15.46:0/1173323895 conn(0x7fe06006ce00 msgr2=0x7fe06006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.869+0000 7fe065b66640 1 -- 172.21.15.46:0/1173323895 shutdown_connections 2024-09-12T03:04:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.869+0000 7fe065b66640 1 -- 172.21.15.46:0/1173323895 wait complete. 2024-09-12T03:04:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.869+0000 7fe065b66640 1 Processor -- start 2024-09-12T03:04:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 -- start start 2024-09-12T03:04:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601091f0 0x7fe0601b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 0x7fe06007a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe06007e590 con 0x7fe0601091f0 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe06007e410 con 0x7fe060109b70 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe065b66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe06007e710 con 0x7fe06010d610 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.870+0000 7fe05effd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50976/0 (socket says 172.21.15.46:50976) 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 -- 172.21.15.46:0/2549008134 learned_addr learned my addr 172.21.15.46:0/2549008134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 0x7fe06007a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601091f0 0x7fe0601b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 -- 172.21.15.46:0/2549008134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 msgr2=0x7fe06007a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 0x7fe06007a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 -- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601091f0 msgr2=0x7fe0601b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601091f0 0x7fe0601b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 -- 172.21.15.46:0/2549008134 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06007aa00 con 0x7fe060109b70 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05ffff640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 0x7fe06007a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.871+0000 7fe05effd640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe050009e30 tx=0x7fe050031ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.872+0000 7fe05cff9640 1 -- 172.21.15.46:0/2549008134 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500053a0 con 0x7fe060109b70 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.872+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06007ac90 con 0x7fe060109b70 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.872+0000 7fe05cff9640 1 -- 172.21.15.46:0/2549008134 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe050032930 con 0x7fe060109b70 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.872+0000 7fe05cff9640 1 -- 172.21.15.46:0/2549008134 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050003e50 con 0x7fe060109b70 2024-09-12T03:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.872+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601c0870 con 0x7fe060109b70 2024-09-12T03:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.875+0000 7fe0689db740 1 Processor -- start 2024-09-12T03:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.876+0000 7fe05cff9640 1 -- 172.21.15.46:0/2549008134 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe05003c410 con 0x7fe060109b70 2024-09-12T03:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 -- start start 2024-09-12T03:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f3613f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f3613fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3613492f0 con 0x55f3613f84c0 2024-09-12T03:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f3612f1270 con 0x55f3613e49f0 2024-09-12T03:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.877+0000 7fe0689db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3613452b0 con 0x55f3613e5310 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50992/0 (socket says 172.21.15.46:50992) 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 -- 172.21.15.46:0/403456137 learned_addr learned my addr 172.21.15.46:0/403456137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05effd640 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f3613fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 -- 172.21.15.46:0/403456137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 msgr2=0x55f3613f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f3613f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 -- 172.21.15.46:0/403456137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 msgr2=0x55f3613fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f3613fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05ffff640 1 -- 172.21.15.46:0/403456137 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3612f0340 con 0x55f3613e49f0 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05effd640 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f3613fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.878+0000 7fe05cff9640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 0x7fe0480a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.879+0000 7fe05ffff640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 0x7fe0480a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.879+0000 7fe05ffff640 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe054011540 tx=0x7fe054011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310444725a43162d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.879+0000 7fe05cff9640 1 -- 172.21.15.46:0/2549008134 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fe0500f03c0 con 0x7fe060109b70 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.880+0000 7fe045ffb640 1 -- 172.21.15.46:0/403456137 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400f800 con 0x55f3613e49f0 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.881+0000 7fe045ffb640 1 -- 172.21.15.46:0/403456137 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05400fe50 con 0x55f3613e49f0 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.881+0000 7fe045ffb640 1 -- 172.21.15.46:0/403456137 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054018410 con 0x55f3613e49f0 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 -- 172.21.15.46:0/403456137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 msgr2=0x55f3613e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe054011540 tx=0x7fe054011a00 comp rx=0 tx=0).stop 2024-09-12T03:04:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe05ffff640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 0x7fe0480a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe054020000 tx=0x7fe05401b040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 -- 172.21.15.46:0/403456137 shutdown_connections 2024-09-12T03:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f3613fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f3613f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 --2- 172.21.15.46:0/403456137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613e4dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 -- 172.21.15.46:0/403456137 >> 172.21.15.46:0/403456137 conn(0x55f3613e1060 msgr2=0x55f3613e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.882+0000 7fe0689db740 1 -- 172.21.15.46:0/403456137 shutdown_connections 2024-09-12T03:04:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.883+0000 7fe0689db740 1 -- 172.21.15.46:0/403456137 wait complete. 2024-09-12T03:04:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.884+0000 7fe0689db740 1 -- start start 2024-09-12T03:04:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.884+0000 7fe0689db740 1 Processor -- start 2024-09-12T03:04:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.884+0000 7fe0689db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613f84c0 0x55f3613f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3613fb1c0 con 0x55f3613e5310 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f3612d2290 con 0x55f3613f84c0 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3613fb340 con 0x55f3613e49f0 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe0689db740 10 client.0 _ll_register_callbacks cb 0x55f361303140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:58686/0 (socket says 172.21.15.46:58686) 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 -- 172.21.15.46:0/1313018446 learned_addr learned my addr 172.21.15.46:0/1313018446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05effd640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05ffff640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613f84c0 0x55f3613f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 -- 172.21.15.46:0/1313018446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613f84c0 msgr2=0x55f3613f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613f84c0 0x55f3613f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 -- 172.21.15.46:0/1313018446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 msgr2=0x55f3613f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05f7fe640 1 -- 172.21.15.46:0/1313018446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36105c210 con 0x55f3613e49f0 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.885+0000 7fe05effd640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe05800fa90 tx=0x7fe05800fe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639474804088db63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe03effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:04:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800eba0 con 0x55f3613e49f0 2024-09-12T03:04:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fe05800ed40 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05400fe50 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe058019420 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe0689db740 10 client.25921 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f361128620 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.886+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05400fe50 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe05800b050 con 0x55f3613e49f0 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 10 client.25921 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 20 client.25921 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f3614085d0 0x55f36140aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 4 client.25921 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f361356a20 con 0x55f3614085d0 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.887+0000 7fe05effd640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f3614085d0 0x55f36140aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.888+0000 7fe05effd640 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f3614085d0 0x55f36140aa20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9292288ea9030694 server_cookie=231990deba86631e in_seq=0 out_seq=0 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.888+0000 7fe03effd640 10 client.25921 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe03effd640 1 -- 172.21.15.46:0/1313018446 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f361356a20 con 0x55f3614085d0 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe03effd640 10 client.25921 handle_command_reply: tid=0 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 client.25921 shutdown 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 msgr2=0x55f3613f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe05800fa90 tx=0x7fe05800fe90 comp rx=0 tx=0).stop 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 shutdown_connections 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f3614085d0 0x55f36140aa20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613f84c0 0x55f3613f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe03effd640 0 client.25921 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 --2- 172.21.15.46:0/1313018446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e49f0 0x55f3613f5240 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 >> 172.21.15.46:0/1313018446 conn(0x55f3613e1060 msgr2=0x55f3613e6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 shutdown_connections 2024-09-12T03:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 1 -- 172.21.15.46:0/1313018446 wait complete. 2024-09-12T03:04:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.889+0000 7fe0689db740 20 client.25921 trim_cache size 0 max 16384 2024-09-12T03:04:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 Processor -- start 2024-09-12T03:04:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 -- start start 2024-09-12T03:04:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f361410330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f361410870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe0689db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c012940 con 0x55f3613f84c0 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.912+0000 7fe05effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f361410330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f361410330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:58696/0 (socket says 172.21.15.46:58696) 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:50996/0 (socket says 172.21.15.46:50996) 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05effd640 1 -- 172.21.15.46:0/3906202149 learned_addr learned my addr 172.21.15.46:0/3906202149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c0101b0 con 0x55f3613e49f0 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00e020 con 0x55f3613e5310 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05ffff640 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f361410870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3906202149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 msgr2=0x55f361410330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f361410330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3906202149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 msgr2=0x55f361410870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f361410870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3906202149 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3613f41d0 con 0x55f3613e49f0 2024-09-12T03:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.913+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe05800c9b0 tx=0x7fe05800ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa6b31f13d95823 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.914+0000 7fe0457fa640 1 -- 172.21.15.46:0/3906202149 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800c410 con 0x55f3613e49f0 2024-09-12T03:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.914+0000 7fe0457fa640 1 -- 172.21.15.46:0/3906202149 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05800c5b0 con 0x55f3613e49f0 2024-09-12T03:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.914+0000 7fe0457fa640 1 -- 172.21.15.46:0/3906202149 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580195c0 con 0x55f3613e49f0 2024-09-12T03:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.914+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 msgr2=0x55f3613fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe05800c9b0 tx=0x7fe05800ce00 comp rx=0 tx=0).stop 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 shutdown_connections 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613f84c0 0x55f361410870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613e5310 0x55f361410330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 --2- 172.21.15.46:0/3906202149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613fa8c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 >> 172.21.15.46:0/3906202149 conn(0x55f3613e1060 msgr2=0x55f3613f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 shutdown_connections 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.915+0000 7fe0689db740 1 -- 172.21.15.46:0/3906202149 wait complete. 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.916+0000 7fe0689db740 1 -- start start 2024-09-12T03:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.916+0000 7fe0689db740 1 Processor -- start 2024-09-12T03:04:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe0689db740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe0689db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe05f7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:51000/0 (socket says 172.21.15.46:51000) 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe05effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe05effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613fa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59508/0 (socket says 172.21.15.46:59508) 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3861224927 learned_addr learned my addr 172.21.15.46:0/3861224927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.917+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613f84c0 0x55f3613fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00e670 con 0x55f3613e5310 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00fae0 con 0x55f3613e49f0 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05ffff640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613f84c0 0x55f3613fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c00ed40 con 0x55f3613f84c0 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613f84c0 msgr2=0x55f3613fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613f84c0 0x55f3613fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3861224927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 msgr2=0x55f3613fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.918+0000 7fe05f7fe640 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3613fec70 con 0x55f3613e49f0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe05f7fe640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe058000f80 tx=0x7fe05800edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffab1e06f7d81b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe0689db740 10 client.0 _ll_register_callbacks cb 0x7fe05800f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe03effd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe03effd640 1 -- 172.21.15.46:0/3861224927 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058037060 con 0x55f3613e49f0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe03effd640 1 -- 172.21.15.46:0/3861224927 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9603+0+0 (secure 0 0 0) 0x7fe05800c9e0 con 0x55f3613e49f0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f361350d70 con 0x55f3613e49f0 2024-09-12T03:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe03effd640 1 -- 172.21.15.46:0/3861224927 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe05803e050 con 0x55f3613e49f0 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe0689db740 10 client.25985 fetch_fsmap learned FSMap version 39 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.919+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55f3614733a0 con 0x55f3613e49f0 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe03effd640 1 -- 172.21.15.46:0/3861224927 <== mon.2 v2:172.21.15.145:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe058030070 con 0x55f3613e49f0 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 10 client.25985 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 20 client.25985 populate_metadata read hostname 'smithi046' 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f361473980 0x55f361475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 4 client.25985 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f361350d70 con 0x55f361473980 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.920+0000 7fe05effd640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f361473980 0x55f361475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.921+0000 7fe05effd640 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f361473980 0x55f361475e00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199e10df4a539b09 server_cookie=e84c02070b05eb70 in_seq=0 out_seq=0 2024-09-12T03:04:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.921+0000 7fe03effd640 10 client.25985 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.947+0000 7fe03effd640 1 -- 172.21.15.46:0/3861224927 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f361350d70 con 0x55f361473980 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.947+0000 7fe03effd640 10 client.25985 handle_command_reply: tid=0 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 client.25985 shutdown 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 msgr2=0x55f3613f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe058000f80 tx=0x7fe05800edb0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 shutdown_connections 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f361473980 0x55f361475e00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe03effd640 0 client.25985 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3613f84c0 0x55f3613fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3613e5310 0x55f3613fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 --2- 172.21.15.46:0/3861224927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f3613e49f0 0x55f3613f9c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 >> 172.21.15.46:0/3861224927 conn(0x55f3613e1060 msgr2=0x55f3613e6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 shutdown_connections 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 1 -- 172.21.15.46:0/3861224927 wait complete. 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.948+0000 7fe0689db740 20 client.25985 trim_cache size 0 max 16384 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:04:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:04:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 msgr2=0x7fe0480a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 0x7fe0480a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe054020000 tx=0x7fe05401b040 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 msgr2=0x7fe0601b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe050009e30 tx=0x7fe050031ef0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 shutdown_connections 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fe0480a3090 0x7fe0480a5550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe06010d610 0x7fe06007a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe060109b70 0x7fe0601b8e90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 --2- 172.21.15.46:0/2549008134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601091f0 0x7fe0601b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 >> 172.21.15.46:0/2549008134 conn(0x7fe06006ce00 msgr2=0x7fe060071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.949+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 shutdown_connections 2024-09-12T03:04:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:04:50.950+0000 7fe065b66640 1 -- 172.21.15.46:0/2549008134 wait complete. 2024-09-12T03:04:52.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:51 smithi196 ceph-mon[34587]: pgmap v461: 129 pgs: 129 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.8 MiB/s wr, 2.22k op/s 2024-09-12T03:04:52.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:51 smithi046 ceph-mon[30951]: pgmap v461: 129 pgs: 129 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.8 MiB/s wr, 2.22k op/s 2024-09-12T03:04:52.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:51 smithi145 ceph-mon[34674]: pgmap v461: 129 pgs: 129 active+clean; 17 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.8 MiB/s wr, 2.22k op/s 2024-09-12T03:04:54.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:53 smithi196 ceph-mon[34587]: pgmap v462: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-09-12T03:04:54.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:53 smithi046 ceph-mon[30951]: pgmap v462: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-09-12T03:04:54.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:53 smithi145 ceph-mon[34674]: pgmap v462: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-09-12T03:04:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:56 smithi196 ceph-mon[34587]: pgmap v463: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 1.54k op/s 2024-09-12T03:04:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:56 smithi196 ceph-mon[34587]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:04:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:56 smithi046 ceph-mon[30951]: pgmap v463: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 1.54k op/s 2024-09-12T03:04:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:56 smithi046 ceph-mon[30951]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:04:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:56 smithi145 ceph-mon[34674]: pgmap v463: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 1.54k op/s 2024-09-12T03:04:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:56 smithi145 ceph-mon[34674]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:04:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: pgmap v465: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.5 MiB/s wr, 1.11k op/s 2024-09-12T03:04:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:58.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:58.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:58.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:04:58 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:04:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: pgmap v465: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.5 MiB/s wr, 1.11k op/s 2024-09-12T03:04:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:58.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:58.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:04:58 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: pgmap v465: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.5 MiB/s wr, 1.11k op/s 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:04:58 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:05:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:00 smithi196 ceph-mon[34587]: pgmap v466: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 853 op/s 2024-09-12T03:05:00.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:00 smithi046 ceph-mon[30951]: pgmap v466: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 853 op/s 2024-09-12T03:05:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:00 smithi145 ceph-mon[34674]: pgmap v466: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 853 op/s 2024-09-12T03:05:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:02 smithi145 ceph-mon[34674]: pgmap v467: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.3 MiB/s wr, 570 op/s 2024-09-12T03:05:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:02 smithi196 ceph-mon[34587]: pgmap v467: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.3 MiB/s wr, 570 op/s 2024-09-12T03:05:02.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:02 smithi046 ceph-mon[30951]: pgmap v467: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.3 MiB/s wr, 570 op/s 2024-09-12T03:05:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 -- 172.21.15.46:0/1545718991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38641138b0 msgr2=0x7f3864115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 --2- 172.21.15.46:0/1545718991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38641138b0 0x7f3864115ca0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ca0 tx=0x7f385c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 -- 172.21.15.46:0/1545718991 shutdown_connections 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 --2- 172.21.15.46:0/1545718991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38641138b0 0x7f3864115ca0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 --2- 172.21.15.46:0/1545718991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f386410acb0 0x7f3864113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 --2- 172.21.15.46:0/1545718991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410a300 0x7f386410a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 -- 172.21.15.46:0/1545718991 >> 172.21.15.46:0/1545718991 conn(0x7f386406ce00 msgr2=0x7f386406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 -- 172.21.15.46:0/1545718991 shutdown_connections 2024-09-12T03:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.095+0000 7f386a2b7640 1 -- 172.21.15.46:0/1545718991 wait complete. 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.096+0000 7f386a2b7640 1 Processor -- start 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.096+0000 7f386a2b7640 1 -- start start 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410acb0 0x7f38641a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38641138b0 0x7f38641ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3864118360 con 0x7f386410a300 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38641181e0 con 0x7f38641138b0 2024-09-12T03:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f386a2b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38641184e0 con 0x7f386410acb0 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f3863fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f3863fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54020/0 (socket says 172.21.15.46:54020) 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f3863fff640 1 -- 172.21.15.46:0/3519859181 learned_addr learned my addr 172.21.15.46:0/3519859181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f3868854640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38641138b0 0x7f38641ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.097+0000 7f38637fe640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410acb0 0x7f38641a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 -- 172.21.15.46:0/3519859181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410acb0 msgr2=0x7f38641a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410acb0 0x7f38641a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 -- 172.21.15.46:0/3519859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38641138b0 msgr2=0x7f38641ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38641138b0 0x7f38641ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 -- 172.21.15.46:0/3519859181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38641ac0a0 con 0x7f386410a300 2024-09-12T03:05:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.098+0000 7f3863fff640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f385400d950 tx=0x7f385400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.099+0000 7f38617fa640 1 -- 172.21.15.46:0/3519859181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854017070 con 0x7f386410a300 2024-09-12T03:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.099+0000 7f38617fa640 1 -- 172.21.15.46:0/3519859181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3854009c30 con 0x7f386410a300 2024-09-12T03:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.099+0000 7f38617fa640 1 -- 172.21.15.46:0/3519859181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385400f620 con 0x7f386410a300 2024-09-12T03:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.099+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38641ac390 con 0x7f386410a300 2024-09-12T03:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.099+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641b3c20 con 0x7f386410a300 2024-09-12T03:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.102+0000 7f386d051740 1 Processor -- start 2024-09-12T03:05:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.103+0000 7f386d051740 1 -- start start 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111419af0 0x559111419ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111424770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55911137e2f0 con 0x559111419af0 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559111326270 con 0x55911141a410 2024-09-12T03:05:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f386d051740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55911137a2b0 con 0x559111424cb0 2024-09-12T03:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f38637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f38637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:56494/0 (socket says 172.21.15.46:56494) 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f38637fe640 1 -- 172.21.15.46:0/358073753 learned_addr learned my addr 172.21.15.46:0/358073753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.104+0000 7f38617fa640 1 -- 172.21.15.46:0/3519859181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f38540109a0 con 0x7f386410a300 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 -- 172.21.15.46:0/358073753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 msgr2=0x559111424770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111424770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 -- 172.21.15.46:0/358073753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111419af0 msgr2=0x559111419ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111419af0 0x559111419ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 -- 172.21.15.46:0/358073753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559111325340 con 0x559111424cb0 2024-09-12T03:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.105+0000 7f38637fe640 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f385800db00 tx=0x7f3858005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908f1ee60631004f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.106+0000 7f384a7fc640 1 -- 172.21.15.46:0/358073753 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858009df0 con 0x559111424cb0 2024-09-12T03:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.106+0000 7f38617fa640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 0x7f384c0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.106+0000 7f384a7fc640 1 -- 172.21.15.46:0/358073753 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3858005020 con 0x559111424cb0 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.106+0000 7f38617fa640 1 -- 172.21.15.46:0/3519859181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f38540cf3d0 con 0x7f386410a300 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.106+0000 7f3868854640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 0x7f384c0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f384a7fc640 1 -- 172.21.15.46:0/358073753 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858004a90 con 0x559111424cb0 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 -- 172.21.15.46:0/358073753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 msgr2=0x5591114270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f385800db00 tx=0x7f3858005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f3868854640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 0x7f384c0a5490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f385c006010 tx=0x7f385c005f80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 -- 172.21.15.46:0/358073753 shutdown_connections 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111424cb0 0x5591114270a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111424770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 --2- 172.21.15.46:0/358073753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111419af0 0x559111419ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 -- 172.21.15.46:0/358073753 >> 172.21.15.46:0/358073753 conn(0x559111416060 msgr2=0x559111416440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 -- 172.21.15.46:0/358073753 shutdown_connections 2024-09-12T03:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.107+0000 7f386d051740 1 -- 172.21.15.46:0/358073753 wait complete. 2024-09-12T03:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 -- start start 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 Processor -- start 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111422050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x559111422590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559111427920 con 0x559111424cb0 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559111307290 con 0x55911141a410 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.108+0000 7f386d051740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559111427aa0 con 0x559111419af0 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f386d051740 10 client.0 _ll_register_callbacks cb 0x559111338140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:56498/0 (socket says 172.21.15.46:56498) 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 -- 172.21.15.46:0/1990019927 learned_addr learned my addr 172.21.15.46:0/1990019927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f38637fe640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111422050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3868854640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x559111422590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 -- 172.21.15.46:0/1990019927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 msgr2=0x559111422050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111422050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 -- 172.21.15.46:0/1990019927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 msgr2=0x559111422590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x559111422590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.109+0000 7f3863fff640 1 -- 172.21.15.46:0/1990019927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559111091210 con 0x559111419af0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.110+0000 7f3868854640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x559111422590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.110+0000 7f3863fff640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3854097910 tx=0x7f385409b0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe0d0c7b9d2ce693 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.110+0000 7f38437fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.111+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854020050 con 0x559111419af0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.111+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f38540cc960 con 0x559111419af0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.111+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3858005020 con 0x559111419af0 2024-09-12T03:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.111+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f38540ccb60 con 0x559111419af0 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.112+0000 7f386d051740 10 client.25939 fetch_fsmap learned FSMap version 40 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.112+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55911115d620 con 0x559111419af0 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3858005020 con 0x559111419af0 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f3854097200 con 0x559111419af0 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 10 client.25939 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:05:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 20 client.25939 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559111434c60 0x559111437050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 4 client.25939 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.113+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55911138ba20 con 0x559111434c60 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.114+0000 7f38637fe640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559111434c60 0x559111437050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.114+0000 7f38437fe640 10 client.25939 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.114+0000 7f38637fe640 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559111434c60 0x559111437050 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bc3d33e5bba601 server_cookie=1043fa7d7cd4f31d in_seq=0 out_seq=0 2024-09-12T03:05:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.115+0000 7f38437fe640 1 -- 172.21.15.46:0/1990019927 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55911138ba20 con 0x559111434c60 2024-09-12T03:05:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.115+0000 7f38437fe640 10 client.25939 handle_command_reply: tid=0 2024-09-12T03:05:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.115+0000 7f386d051740 1 client.25939 shutdown 2024-09-12T03:05:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.115+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 msgr2=0x559111421b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.115+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3854097910 tx=0x7f385409b0f0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 shutdown_connections 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x559111434c60 0x559111437050 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x559111422590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55911141a410 0x559111422050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f38437fe640 0 client.25939 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 --2- 172.21.15.46:0/1990019927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559111419af0 0x559111421b10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 >> 172.21.15.46:0/1990019927 conn(0x559111416060 msgr2=0x55911141bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 shutdown_connections 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 1 -- 172.21.15.46:0/1990019927 wait complete. 2024-09-12T03:05:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.116+0000 7f386d051740 20 client.25939 trim_cache size 0 max 16384 2024-09-12T03:05:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.143+0000 7f386d051740 1 Processor -- start 2024-09-12T03:05:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.143+0000 7f386d051740 1 -- start start 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911143c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911143ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38641c0700 con 0x559111424cb0 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38641bdfa0 con 0x559111419af0 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f386d051740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38641bbe10 con 0x55911141a410 2024-09-12T03:05:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:40296/0 (socket says 172.21.15.46:40296) 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 -- 172.21.15.46:0/3101428836 learned_addr learned my addr 172.21.15.46:0/3101428836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f38637fe640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911143c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3868854640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911143ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 -- 172.21.15.46:0/3101428836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 msgr2=0x55911143c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911143c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 -- 172.21.15.46:0/3101428836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 msgr2=0x55911143ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911143ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3863fff640 1 -- 172.21.15.46:0/3101428836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55911141a240 con 0x559111419af0 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.144+0000 7f3868854640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911143ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.145+0000 7f3863fff640 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f385409b160 tx=0x7f385400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38181ba48d506cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.145+0000 7f3849ffb640 1 -- 172.21.15.46:0/3101428836 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38540b4460 con 0x559111419af0 2024-09-12T03:05:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.145+0000 7f3849ffb640 1 -- 172.21.15.46:0/3101428836 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3854015070 con 0x559111419af0 2024-09-12T03:05:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.145+0000 7f3849ffb640 1 -- 172.21.15.46:0/3101428836 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385409b4b0 con 0x559111419af0 2024-09-12T03:05:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.146+0000 7f386d051740 1 -- 172.21.15.46:0/3101428836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 msgr2=0x5591114223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.146+0000 7f386d051740 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f385409b160 tx=0x7f385400bf60 comp rx=0 tx=0).stop 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.147+0000 7f386d051740 1 -- 172.21.15.46:0/3101428836 shutdown_connections 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.147+0000 7f386d051740 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911143ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.147+0000 7f386d051740 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911143c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.147+0000 7f386d051740 1 --2- 172.21.15.46:0/3101428836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x5591114223c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.147+0000 7f386d051740 1 -- 172.21.15.46:0/3101428836 >> 172.21.15.46:0/3101428836 conn(0x559111416060 msgr2=0x559111423b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.149+0000 7f386d051740 1 -- 172.21.15.46:0/3101428836 shutdown_connections 2024-09-12T03:05:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.149+0000 7f386d051740 1 -- 172.21.15.46:0/3101428836 wait complete. 2024-09-12T03:05:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.150+0000 7f386d051740 1 -- start start 2024-09-12T03:05:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.150+0000 7f386d051740 1 Processor -- start 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x55911142d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911141e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38641bc460 con 0x559111424cb0 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38641bd8d0 con 0x559111419af0 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38641bcb30 con 0x55911141a410 2024-09-12T03:05:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f386d051740 10 client.0 _ll_register_callbacks cb 0x7f385409b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54048/0 (socket says 172.21.15.46:54048) 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 -- 172.21.15.46:0/1720266933 learned_addr learned my addr 172.21.15.46:0/1720266933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f38637fe640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911141e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3863fff640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x55911142d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 -- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 msgr2=0x55911141e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911141e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 -- 172.21.15.46:0/1720266933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 msgr2=0x55911142d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.151+0000 7f3868854640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x55911142d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f3868854640 1 -- 172.21.15.46:0/1720266933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55911141f6d0 con 0x559111424cb0 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f3863fff640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x55911142d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f3868854640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ca0 tx=0x7f385c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c05af738295732 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f38437fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004450 con 0x559111424cb0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f385c0045f0 con 0x559111424cb0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.152+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3854015070 con 0x559111424cb0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f385c05c050 con 0x559111424cb0 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 10 client.16395 fetch_fsmap learned FSMap version 40 2024-09-12T03:05:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5591114b0940 con 0x559111424cb0 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3854015070 con 0x559111424cb0 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f385c051020 con 0x559111424cb0 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 10 client.16395 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 20 client.16395 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5591114b1120 0x5591114b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 4 client.16395 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:05:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.153+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559111385d70 con 0x5591114b1120 2024-09-12T03:05:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.154+0000 7f3863fff640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5591114b1120 0x5591114b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.154+0000 7f3863fff640 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5591114b1120 0x5591114b3540 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af35fb5ac53cfa38 server_cookie=d5db465d066a6d9 in_seq=0 out_seq=0 2024-09-12T03:05:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.154+0000 7f38437fe640 10 client.16395 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f38437fe640 1 -- 172.21.15.46:0/1720266933 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559111385d70 con 0x5591114b1120 2024-09-12T03:05:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f38437fe640 10 client.16395 handle_command_reply: tid=0 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 client.16395 shutdown 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 msgr2=0x55911141eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ca0 tx=0x7f385c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 shutdown_connections 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x5591114b1120 0x5591114b3540 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559111424cb0 0x55911141eea0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55911141a410 0x55911141e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 --2- 172.21.15.46:0/1720266933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559111419af0 0x55911142d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 >> 172.21.15.46:0/1720266933 conn(0x559111416060 msgr2=0x55911141b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.169+0000 7f38437fe640 0 client.16395 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.170+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 shutdown_connections 2024-09-12T03:05:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.170+0000 7f386d051740 1 -- 172.21.15.46:0/1720266933 wait complete. 2024-09-12T03:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.170+0000 7f386d051740 20 client.16395 trim_cache size 0 max 16384 2024-09-12T03:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 msgr2=0x7f384c0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 0x7f384c0a5490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f385c006010 tx=0x7f385c005f80 comp rx=0 tx=0).stop 2024-09-12T03:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 msgr2=0x7f38641a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f385400d950 tx=0x7f385400de10 comp rx=0 tx=0).stop 2024-09-12T03:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 shutdown_connections 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f384c0a2fd0 0x7f384c0a5490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38641138b0 0x7f38641ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.171+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386410acb0 0x7f38641a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.172+0000 7f386a2b7640 1 --2- 172.21.15.46:0/3519859181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386410a300 0x7f38641a70b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.172+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 >> 172.21.15.46:0/3519859181 conn(0x7f386406ce00 msgr2=0x7f3864071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.172+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 shutdown_connections 2024-09-12T03:05:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:04.172+0000 7f386a2b7640 1 -- 172.21.15.46:0/3519859181 wait complete. 2024-09-12T03:05:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:04 smithi196 ceph-mon[34587]: pgmap v468: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 358 op/s 2024-09-12T03:05:04.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:04 smithi046 ceph-mon[30951]: pgmap v468: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 358 op/s 2024-09-12T03:05:04.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:04 smithi145 ceph-mon[34674]: pgmap v468: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 358 op/s 2024-09-12T03:05:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:06 smithi196 ceph-mon[34587]: pgmap v469: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-12T03:05:06.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:06 smithi046 ceph-mon[30951]: pgmap v469: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-12T03:05:06.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:06 smithi145 ceph-mon[34674]: pgmap v469: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-12T03:05:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:08 smithi196 ceph-mon[34587]: pgmap v470: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-09-12T03:05:08.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:08 smithi046 ceph-mon[30951]: pgmap v470: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-09-12T03:05:08.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:08 smithi145 ceph-mon[34674]: pgmap v470: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-09-12T03:05:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:10 smithi196 ceph-mon[34587]: pgmap v471: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 408 op/s 2024-09-12T03:05:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:10 smithi046 ceph-mon[30951]: pgmap v471: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 408 op/s 2024-09-12T03:05:10.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:10 smithi145 ceph-mon[34674]: pgmap v471: 129 pgs: 129 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.5 MiB/s wr, 408 op/s 2024-09-12T03:05:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:12 smithi196 ceph-mon[34587]: pgmap v472: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 364 op/s 2024-09-12T03:05:12.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:12 smithi046 ceph-mon[30951]: pgmap v472: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 364 op/s 2024-09-12T03:05:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:12 smithi145 ceph-mon[34674]: pgmap v472: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 364 op/s 2024-09-12T03:05:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:14 smithi196 ceph-mon[34587]: pgmap v473: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 434 op/s 2024-09-12T03:05:14.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:14 smithi046 ceph-mon[30951]: pgmap v473: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 434 op/s 2024-09-12T03:05:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:14 smithi145 ceph-mon[34674]: pgmap v473: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 434 op/s 2024-09-12T03:05:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:16 smithi196 ceph-mon[34587]: pgmap v474: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-09-12T03:05:16.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:16 smithi046 ceph-mon[30951]: pgmap v474: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-09-12T03:05:16.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:16 smithi145 ceph-mon[34674]: pgmap v474: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-09-12T03:05:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 -- 172.21.15.46:0/1081669168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab410acb0 msgr2=0x7faab4113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 --2- 172.21.15.46:0/1081669168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab410acb0 0x7faab4113370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009860 tx=0x7faaa402a340 comp rx=0 tx=0).stop 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 -- 172.21.15.46:0/1081669168 shutdown_connections 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 --2- 172.21.15.46:0/1081669168 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab41138b0 0x7faab4115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 --2- 172.21.15.46:0/1081669168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab410acb0 0x7faab4113370 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009860 tx=0x7faaa402a340 comp rx=0 tx=0).stop 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 --2- 172.21.15.46:0/1081669168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab410a300 0x7faab410a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.316+0000 7faabc2e6640 1 -- 172.21.15.46:0/1081669168 >> 172.21.15.46:0/1081669168 conn(0x7faab406ce00 msgr2=0x7faab406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.317+0000 7faabc2e6640 1 -- 172.21.15.46:0/1081669168 shutdown_connections 2024-09-12T03:05:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.317+0000 7faabc2e6640 1 -- 172.21.15.46:0/1081669168 wait complete. 2024-09-12T03:05:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.318+0000 7faabc2e6640 1 Processor -- start 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.318+0000 7faabc2e6640 1 -- start start 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.318+0000 7faabc2e6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab410a300 0x7faab4110880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faabc2e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 0x7faab4110dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faabc2e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faabc2e6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab41181e0 con 0x7faab41138b0 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faabc2e6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faab4118060 con 0x7faab410a300 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faabc2e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faab4118360 con 0x7faab4111300 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba082640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab410a300 0x7faab4110880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:56996/0 (socket says 172.21.15.46:56996) 2024-09-12T03:05:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 -- 172.21.15.46:0/4092752230 learned_addr learned my addr 172.21.15.46:0/4092752230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faab9881640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 0x7faab4110dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 -- 172.21.15.46:0/4092752230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab410a300 msgr2=0x7faab4110880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab410a300 0x7faab4110880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 -- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 msgr2=0x7faab4110dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 0x7faab4110dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.319+0000 7faaba883640 1 -- 172.21.15.46:0/4092752230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab4124390 con 0x7faab4111300 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faab9881640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 0x7faab4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faaba883640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faaac00ef30 tx=0x7faaac00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faaab7fe640 1 -- 172.21.15.46:0/4092752230 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac004b20 con 0x7faab4111300 2024-09-12T03:05:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab4124680 con 0x7faab4111300 2024-09-12T03:05:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab4124ba0 con 0x7faab4111300 2024-09-12T03:05:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.320+0000 7faaab7fe640 1 -- 172.21.15.46:0/4092752230 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac003e20 con 0x7faab4111300 2024-09-12T03:05:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.321+0000 7faaab7fe640 1 -- 172.21.15.46:0/4092752230 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac004800 con 0x7faab4111300 2024-09-12T03:05:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.324+0000 7faaab7fe640 1 -- 172.21.15.46:0/4092752230 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faaac01d020 con 0x7faab4111300 2024-09-12T03:05:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.325+0000 7faabf07d740 1 Processor -- start 2024-09-12T03:05:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.326+0000 7faabf07d740 1 -- start start 2024-09-12T03:05:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faaab7fe640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 0x7faaa00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed84770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55578ecde2f0 con 0x55578ed79af0 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55578ec86270 con 0x55578ed7a410 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faabf07d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55578ecda2b0 con 0x55578ed84cb0 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faaab7fe640 1 -- 172.21.15.46:0/4092752230 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7faaac0cf610 con 0x7faab4111300 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.327+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faab9881640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 0x7faaa00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:57012/0 (socket says 172.21.15.46:57012) 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 -- 172.21.15.46:0/2336905860 learned_addr learned my addr 172.21.15.46:0/2336905860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faab9881640 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed84770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 -- 172.21.15.46:0/2336905860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 msgr2=0x55578ed84770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed84770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 -- 172.21.15.46:0/2336905860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 msgr2=0x55578ed79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 -- 172.21.15.46:0/2336905860 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578ec85340 con 0x55578ed84cb0 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.328+0000 7faaba883640 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faaac0a3e30 tx=0x7faaac09bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f953a772e0e21220 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.329+0000 7faaa8bfe640 1 -- 172.21.15.46:0/2336905860 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac01a070 con 0x55578ed84cb0 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.329+0000 7faaa8bfe640 1 -- 172.21.15.46:0/2336905860 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac0a4360 con 0x55578ed84cb0 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.329+0000 7faaa8bfe640 1 -- 172.21.15.46:0/2336905860 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0ccba0 con 0x55578ed84cb0 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 -- 172.21.15.46:0/2336905860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 msgr2=0x55578ed870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faaac0a3e30 tx=0x7faaac09bf30 comp rx=0 tx=0).stop 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faab9881640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 0x7faaa00a5360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaa4035040 tx=0x7faaa4005a90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 -- 172.21.15.46:0/2336905860 shutdown_connections 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed870a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed84770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 --2- 172.21.15.46:0/2336905860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 -- 172.21.15.46:0/2336905860 >> 172.21.15.46:0/2336905860 conn(0x55578ed76060 msgr2=0x55578ed76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 -- 172.21.15.46:0/2336905860 shutdown_connections 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.330+0000 7faabf07d740 1 -- 172.21.15.46:0/2336905860 wait complete. 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 -- start start 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 Processor -- start 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55578ed87880 con 0x55578ed79af0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55578ec67290 con 0x55578ed7a410 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55578ed87a00 con 0x55578ed84cb0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.331+0000 7faabf07d740 10 client.0 _ll_register_callbacks cb 0x55578ec98140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:57028/0 (socket says 172.21.15.46:57028) 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faaba883640 1 -- 172.21.15.46:0/2089722072 learned_addr learned my addr 172.21.15.46:0/2089722072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faaba082640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 -- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 msgr2=0x55578ed82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 -- 172.21.15.46:0/2089722072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 msgr2=0x55578ed81b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed81b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faab9881640 1 -- 172.21.15.46:0/2089722072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578e9f1210 con 0x55578ed7a410 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.332+0000 7faaba883640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faab9881640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed820c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faaa4008000 tx=0x7faaa402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56aaaf21e4f66553 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faa9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa404e030 con 0x55578ed7a410 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7faaa4003d30 con 0x55578ed7a410 2024-09-12T03:05:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac0a4360 con 0x55578ed7a410 2024-09-12T03:05:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7faaa4046040 con 0x55578ed7a410 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faabf07d740 10 client.26012 fetch_fsmap learned FSMap version 40 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.333+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55578eabd620 con 0x55578ed7a410 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac0a4360 con 0x55578ed7a410 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7faaa4049030 con 0x55578ed7a410 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 10 client.26012 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 20 client.26012 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ed95140 0x55578ed975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 4 client.26012 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578eceba20 con 0x55578ed95140 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.334+0000 7faaba082640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ed95140 0x55578ed975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.335+0000 7faaba082640 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ed95140 0x55578ed975c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b69d1e45306c9a server_cookie=3901be325ed58fdf in_seq=0 out_seq=0 2024-09-12T03:05:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.335+0000 7faa9dffb640 10 client.26012 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.336+0000 7faa9dffb640 1 -- 172.21.15.46:0/2089722072 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55578eceba20 con 0x55578ed95140 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.336+0000 7faa9dffb640 10 client.26012 handle_command_reply: tid=0 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.336+0000 7faabf07d740 1 client.26012 shutdown 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.336+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 msgr2=0x55578ed820c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.336+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed820c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faaa4008000 tx=0x7faaa402a340 comp rx=0 tx=0).stop 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 shutdown_connections 2024-09-12T03:05:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ed95140 0x55578ed975c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed820c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 --2- 172.21.15.46:0/2089722072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed81b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 >> 172.21.15.46:0/2089722072 conn(0x55578ed76060 msgr2=0x55578ed7bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faa9dffb640 0 client.26012 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 shutdown_connections 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 1 -- 172.21.15.46:0/2089722072 wait complete. 2024-09-12T03:05:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.338+0000 7faabf07d740 20 client.26012 trim_cache size 0 max 16384 2024-09-12T03:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.364+0000 7faabf07d740 1 Processor -- start 2024-09-12T03:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.364+0000 7faabf07d740 1 -- start start 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed79af0 0x55578ed7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed9ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa98012820 con 0x55578ed7a410 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa980100c0 con 0x55578ed79af0 2024-09-12T03:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faabf07d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800df30 con 0x55578ed84cb0 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faab9881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faab9881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faaba883640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed9ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faab9881640 1 -- 172.21.15.46:0/3284154304 learned_addr learned my addr 172.21.15.46:0/3284154304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.365+0000 7faaba082640 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed79af0 0x55578ed7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 -- 172.21.15.46:0/3284154304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed79af0 msgr2=0x55578ed7c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed79af0 0x55578ed7c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 -- 172.21.15.46:0/3284154304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 msgr2=0x55578ed82430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 -- 172.21.15.46:0/3284154304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578ed97c40 con 0x55578ed84cb0 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faab9881640 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faaba883640 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed9ced0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faaac00ef00 tx=0x7faaac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f03daf5d05bc15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faa9ffff640 1 -- 172.21.15.46:0/3284154304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0b8070 con 0x55578ed84cb0 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.366+0000 7faa9ffff640 1 -- 172.21.15.46:0/3284154304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac084e50 con 0x55578ed84cb0 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.367+0000 7faa9ffff640 1 -- 172.21.15.46:0/3284154304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac01a040 con 0x55578ed84cb0 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.367+0000 7faabf07d740 1 -- 172.21.15.46:0/3284154304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 msgr2=0x55578ed9ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.367+0000 7faabf07d740 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed9ced0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faaac00ef00 tx=0x7faaac004a40 comp rx=0 tx=0).stop 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 -- 172.21.15.46:0/3284154304 shutdown_connections 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed9ced0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed7a410 0x55578ed82430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 --2- 172.21.15.46:0/3284154304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed79af0 0x55578ed7c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 -- 172.21.15.46:0/3284154304 >> 172.21.15.46:0/3284154304 conn(0x55578ed76060 msgr2=0x55578ed83c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 -- 172.21.15.46:0/3284154304 shutdown_connections 2024-09-12T03:05:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.368+0000 7faabf07d740 1 -- 172.21.15.46:0/3284154304 wait complete. 2024-09-12T03:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 -- start start 2024-09-12T03:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 Processor -- start 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed88600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800e580 con 0x55578ed79af0 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800f9f0 con 0x55578ed7a410 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800ec50 con 0x55578ed84cb0 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faabf07d740 10 client.0 _ll_register_callbacks cb 0x7faaa4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faaba082640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faaba082640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42444/0 (socket says 172.21.15.46:42444) 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faaba082640 1 -- 172.21.15.46:0/4272856409 learned_addr learned my addr 172.21.15.46:0/4272856409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faaba883640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.369+0000 7faab9881640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed88600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 -- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 msgr2=0x55578ed88b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed88b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 -- 172.21.15.46:0/4272856409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 msgr2=0x55578ed88600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed88600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 -- 172.21.15.46:0/4272856409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578ed84710 con 0x55578ed79af0 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba883640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faaba082640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55578ed97e70 tx=0x7faab0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6535b857bf12103 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faa9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004460 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7faab00035f0 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.370+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac084e50 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7faab002e050 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 10 client.16410 fetch_fsmap learned FSMap version 40 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55578ed75e90 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab0003ca0 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7faab0026400 con 0x55578ed79af0 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 10 client.16410 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 20 client.16410 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ee00230 0x55578ee02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 4 client.16410 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.371+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7faaac084e50 con 0x55578ee00230 2024-09-12T03:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.372+0000 7faab9881640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ee00230 0x55578ee02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.372+0000 7faa9dffb640 10 client.16410 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.372+0000 7faab9881640 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ee00230 0x55578ee02650 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c0ad4922603321 server_cookie=70286fc41ce355f2 in_seq=0 out_seq=0 2024-09-12T03:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.385+0000 7faa9dffb640 1 -- 172.21.15.46:0/4272856409 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7faaac084e50 con 0x55578ee00230 2024-09-12T03:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.385+0000 7faa9dffb640 10 client.16410 handle_command_reply: tid=0 2024-09-12T03:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 client.16410 shutdown 2024-09-12T03:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 msgr2=0x55578ed8d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55578ed97e70 tx=0x7faab0010800 comp rx=0 tx=0).stop 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 shutdown_connections 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55578ee00230 0x55578ee02650 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faa9dffb640 0 client.16410 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55578ed84cb0 0x55578ed88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55578ed7a410 0x55578ed88600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 --2- 172.21.15.46:0/4272856409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ed79af0 0x55578ed8d2e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.386+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 >> 172.21.15.46:0/4272856409 conn(0x55578ed76060 msgr2=0x55578ed833e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.387+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 shutdown_connections 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.387+0000 7faabf07d740 1 -- 172.21.15.46:0/4272856409 wait complete. 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.387+0000 7faabf07d740 20 client.16410 trim_cache size 0 max 16384 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.390+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 msgr2=0x7faaa00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.390+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 0x7faaa00a5360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaa4035040 tx=0x7faaa4005a90 comp rx=0 tx=0).stop 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.390+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 msgr2=0x7faab4123bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.390+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faaac00ef30 tx=0x7faaac00c800 comp rx=0 tx=0).stop 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 shutdown_connections 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7faaa00a2ea0 0x7faaa00a5360 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab4111300 0x7faab4123bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab41138b0 0x7faab4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 --2- 172.21.15.46:0/4092752230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faab410a300 0x7faab4110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.391+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 >> 172.21.15.46:0/4092752230 conn(0x7faab406ce00 msgr2=0x7faab4071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.392+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 shutdown_connections 2024-09-12T03:05:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:17.392+0000 7faabc2e6640 1 -- 172.21.15.46:0/4092752230 wait complete. 2024-09-12T03:05:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:18 smithi196 ceph-mon[34587]: pgmap v475: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 377 op/s 2024-09-12T03:05:18.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:18 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:18.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:18 smithi046 ceph-mon[30951]: pgmap v475: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 377 op/s 2024-09-12T03:05:18.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:18 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:18.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:18 smithi145 ceph-mon[34674]: pgmap v475: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 377 op/s 2024-09-12T03:05:18.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:18 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:20.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:20 smithi196 ceph-mon[34587]: pgmap v476: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 402 op/s 2024-09-12T03:05:20.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:20 smithi046 ceph-mon[30951]: pgmap v476: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 402 op/s 2024-09-12T03:05:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:20 smithi145 ceph-mon[34674]: pgmap v476: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 402 op/s 2024-09-12T03:05:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:22 smithi196 ceph-mon[34587]: pgmap v477: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 337 op/s 2024-09-12T03:05:22.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:22 smithi046 ceph-mon[30951]: pgmap v477: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 337 op/s 2024-09-12T03:05:22.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:22 smithi145 ceph-mon[34674]: pgmap v477: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 337 op/s 2024-09-12T03:05:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:24 smithi196 ceph-mon[34587]: pgmap v478: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 421 op/s 2024-09-12T03:05:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:24 smithi046 ceph-mon[30951]: pgmap v478: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 421 op/s 2024-09-12T03:05:24.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:24 smithi145 ceph-mon[34674]: pgmap v478: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.6 MiB/s wr, 421 op/s 2024-09-12T03:05:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:26 smithi196 ceph-mon[34587]: pgmap v479: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-09-12T03:05:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:26 smithi046 ceph-mon[30951]: pgmap v479: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-09-12T03:05:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:26 smithi145 ceph-mon[34674]: pgmap v479: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-09-12T03:05:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:28 smithi196 ceph-mon[34587]: pgmap v480: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-09-12T03:05:28.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:28 smithi046 ceph-mon[30951]: pgmap v480: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-09-12T03:05:28.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:28 smithi145 ceph-mon[34674]: pgmap v480: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-09-12T03:05:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 -- 172.21.15.46:0/2516632619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac108360 msgr2=0x7f54ac108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/2516632619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac108360 0x7f54ac108740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f549c009860 tx=0x7f549c02a340 comp rx=0 tx=0).stop 2024-09-12T03:05:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 -- 172.21.15.46:0/2516632619 shutdown_connections 2024-09-12T03:05:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/2516632619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac10d4b0 0x7f54ac115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/2516632619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac108d10 0x7f54ac10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/2516632619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac108360 0x7f54ac108740 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 -- 172.21.15.46:0/2516632619 >> 172.21.15.46:0/2516632619 conn(0x7f54ac077ad0 msgr2=0x7f54ac079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 -- 172.21.15.46:0/2516632619 shutdown_connections 2024-09-12T03:05:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.531+0000 7f54b1e2e640 1 -- 172.21.15.46:0/2516632619 wait complete. 2024-09-12T03:05:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.532+0000 7f54b1e2e640 1 Processor -- start 2024-09-12T03:05:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.532+0000 7f54b1e2e640 1 -- start start 2024-09-12T03:05:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 0x7f54ac1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 0x7f54ac1a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac118380 con 0x7f54ac10d4b0 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac118200 con 0x7f54ac19e690 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54b1e2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac118500 con 0x7f54ac108360 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 0x7f54ac1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 0x7f54ac1a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 0x7f54ac1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:50166/0 (socket says 172.21.15.46:50166) 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 0x7f54ac1a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58836/0 (socket says 172.21.15.46:58836) 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54ab7fe640 1 -- 172.21.15.46:0/3514975574 learned_addr learned my addr 172.21.15.46:0/3514975574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:51004/0 (socket says 172.21.15.46:51004) 2024-09-12T03:05:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 -- 172.21.15.46:0/3514975574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 msgr2=0x7f54ac1a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 0x7f54ac1a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 -- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 msgr2=0x7f54ac1a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.533+0000 7f54abfff640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 0x7f54ac1a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54abfff640 1 -- 172.21.15.46:0/3514975574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac19f290 con 0x7f54ac19e690 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54abfff640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f54a400ee70 tx=0x7f54a400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54a8ff9640 1 -- 172.21.15.46:0/3514975574 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40099a0 con 0x7f54ac19e690 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54a8ff9640 1 -- 172.21.15.46:0/3514975574 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a4003970 con 0x7f54ac19e690 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54a8ff9640 1 -- 172.21.15.46:0/3514975574 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004750 con 0x7f54ac19e690 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac1ab370 con 0x7f54ac19e690 2024-09-12T03:05:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.534+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1ab720 con 0x7f54ac19e690 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.538+0000 7f54a8ff9640 1 -- 172.21.15.46:0/3514975574 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f54a401d020 con 0x7f54ac19e690 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 Processor -- start 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 -- start start 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a8af83c2f0 con 0x55a8af8d7af0 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a8af7e4270 con 0x55a8af8e2cb0 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8af8382b0 con 0x55a8af8d8410 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58840/0 (socket says 172.21.15.46:58840) 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54aaffd640 1 -- 172.21.15.46:0/665692685 learned_addr learned my addr 172.21.15.46:0/665692685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.539+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54a8ff9640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 0x7f54940a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 -- 172.21.15.46:0/665692685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 msgr2=0x55a8af8e2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 -- 172.21.15.46:0/665692685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 msgr2=0x55a8af8e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 -- 172.21.15.46:0/665692685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8af7e3340 con 0x55a8af8d7af0 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54abfff640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 0x7f54940a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54aaffd640 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009d60 tx=0x7f54a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e45c24cd6171d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f54a8ff9640 1 -- 172.21.15.46:0/3514975574 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f54a40cf520 con 0x7f54ac19e690 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f5491ffb640 1 -- 172.21.15.46:0/665692685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a001e070 con 0x55a8af8d7af0 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f5491ffb640 1 -- 172.21.15.46:0/665692685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a0004250 con 0x55a8af8d7af0 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.540+0000 7f5491ffb640 1 -- 172.21.15.46:0/665692685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0019690 con 0x55a8af8d7af0 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 -- 172.21.15.46:0/665692685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 msgr2=0x55a8af8d7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54abfff640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 0x7f54940a5440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a409b260 tx=0x7f54a400cf70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009d60 tx=0x7f54a0011810 comp rx=0 tx=0).stop 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 -- 172.21.15.46:0/665692685 shutdown_connections 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/665692685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8d7ed0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 -- 172.21.15.46:0/665692685 >> 172.21.15.46:0/665692685 conn(0x55a8af8d4060 msgr2=0x55a8af8d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 -- 172.21.15.46:0/665692685 shutdown_connections 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.541+0000 7f54b4bc9740 1 -- 172.21.15.46:0/665692685 wait complete. 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 -- start start 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 Processor -- start 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a8af8e5920 con 0x55a8af8d7af0 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a8af7c5290 con 0x55a8af8e2cb0 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8af8e5aa0 con 0x55a8af8d8410 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.542+0000 7f54b4bc9740 10 client.0 _ll_register_callbacks cb 0x55a8af7f6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58854/0 (socket says 172.21.15.46:58854) 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54aaffd640 1 -- 172.21.15.46:0/330341513 learned_addr learned my addr 172.21.15.46:0/330341513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54aaffd640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 -- 172.21.15.46:0/330341513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 msgr2=0x55a8af8e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 -- 172.21.15.46:0/330341513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 msgr2=0x55a8af8dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 -- 172.21.15.46:0/330341513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8af54f210 con 0x55a8af8d8410 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54aaffd640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e0140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f549c02a930 tx=0x7f549c004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603cbb7ae57f8bde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.543+0000 7f548affd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c037040 con 0x55a8af8d8410 2024-09-12T03:05:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f549c030650 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a0004250 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f549c030850 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f54b4bc9740 10 client.25975 fetch_fsmap learned FSMap version 41 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a8af61b620 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a0004250 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.544+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f549c0324d0 con 0x55a8af8d8410 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 10 client.25975 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 20 client.25975 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af8f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 4 client.25975 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8af849a20 con 0x55a8af8f2cf0 2024-09-12T03:05:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f54aaffd640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af8f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.545+0000 7f548affd640 10 client.25975 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.546+0000 7f54aaffd640 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af8f50e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff684f4d34bf6d06 server_cookie=a904e2575b3eacb5 in_seq=0 out_seq=0 2024-09-12T03:05:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.546+0000 7f548affd640 1 -- 172.21.15.46:0/330341513 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8af849a20 con 0x55a8af8f2cf0 2024-09-12T03:05:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.546+0000 7f548affd640 10 client.25975 handle_command_reply: tid=0 2024-09-12T03:05:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.546+0000 7f54b4bc9740 1 client.25975 shutdown 2024-09-12T03:05:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 msgr2=0x55a8af8e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e0140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f549c02a930 tx=0x7f549c004250 comp rx=0 tx=0).stop 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 shutdown_connections 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af8f50e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f548affd640 0 client.25975 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e0140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/330341513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 >> 172.21.15.46:0/330341513 conn(0x55a8af8d4060 msgr2=0x55a8af8d9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.547+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 shutdown_connections 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.548+0000 7f54b4bc9740 1 -- 172.21.15.46:0/330341513 wait complete. 2024-09-12T03:05:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.548+0000 7f54b4bc9740 20 client.25975 trim_cache size 0 max 16384 2024-09-12T03:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.568+0000 7f54b4bc9740 1 Processor -- start 2024-09-12T03:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.568+0000 7f54b4bc9740 1 -- start start 2024-09-12T03:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.568+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8da560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.568+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8da560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58862/0 (socket says 172.21.15.46:58862) 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b7cc0 con 0x55a8af8d7af0 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b5560 con 0x55a8af8e2cb0 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b33d0 con 0x55a8af8d8410 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54aaffd640 1 -- 172.21.15.46:0/3478343698 learned_addr learned my addr 172.21.15.46:0/3478343698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54aaffd640 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 -- 172.21.15.46:0/3478343698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 msgr2=0x55a8af8fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 -- 172.21.15.46:0/3478343698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 msgr2=0x55a8af8da560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8da560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54ab7fe640 1 -- 172.21.15.46:0/3478343698 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8af8d3b00 con 0x55a8af8d8410 2024-09-12T03:05:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.569+0000 7f54aaffd640 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.570+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e04b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f549c032ae0 tx=0x7f549c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1b5bffc9fdac0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.570+0000 7f54917fa640 1 -- 172.21.15.46:0/3478343698 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c037030 con 0x55a8af8d8410 2024-09-12T03:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.570+0000 7f54917fa640 1 -- 172.21.15.46:0/3478343698 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f549c038040 con 0x55a8af8d8410 2024-09-12T03:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.570+0000 7f54917fa640 1 -- 172.21.15.46:0/3478343698 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003850 con 0x55a8af8d8410 2024-09-12T03:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- 172.21.15.46:0/3478343698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 msgr2=0x55a8af8e04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e04b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f549c032ae0 tx=0x7f549c02a340 comp rx=0 tx=0).stop 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- 172.21.15.46:0/3478343698 shutdown_connections 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8e2cb0 0x55a8af8fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8d8410 0x55a8af8e04b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/3478343698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d7af0 0x55a8af8da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- 172.21.15.46:0/3478343698 >> 172.21.15.46:0/3478343698 conn(0x55a8af8d4060 msgr2=0x55a8af8e1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- 172.21.15.46:0/3478343698 shutdown_connections 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- 172.21.15.46:0/3478343698 wait complete. 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 -- start start 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.571+0000 7f54b4bc9740 1 Processor -- start 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d8410 0x55a8af8e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b3a20 con 0x55a8af8d8410 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b4e90 con 0x55a8af8d7af0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b40f0 con 0x55a8af8e2cb0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54b4bc9740 10 client.0 _ll_register_callbacks cb 0x7f54a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:51044/0 (socket says 172.21.15.46:51044) 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 -- 172.21.15.46:0/640537520 learned_addr learned my addr 172.21.15.46:0/640537520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d8410 0x55a8af8e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 -- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8e2cb0 msgr2=0x55a8af8e6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.572+0000 7f54aaffd640 1 -- 172.21.15.46:0/640537520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d8410 msgr2=0x55a8af8e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54aaffd640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d8410 0x55a8af8e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54aaffd640 1 -- 172.21.15.46:0/640537520 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8af8e7160 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54aaffd640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096d0 tx=0x7f54a0009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1d660e01b95992 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f548affd640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a002c070 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mon.2 v2:172.21.15.145:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f54a00035f0 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f549c038040 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f54a003f050 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54b4bc9740 10 client.26030 fetch_fsmap learned FSMap version 41 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.573+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a8af95d750 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mon.2 v2:172.21.15.145:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f549c038040 con 0x55a8af8d7af0 2024-09-12T03:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mon.2 v2:172.21.15.145:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f54a0004bf0 con 0x55a8af8d7af0 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 10 client.26030 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 20 client.26030 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af95fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 4 client.26030 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8af843d70 con 0x55a8af8f2cf0 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af95fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.574+0000 7f548affd640 10 client.26030 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.575+0000 7f54ab7fe640 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af95fdf0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca4455a76fc19f0 server_cookie=ebc13d9f0867f0ec in_seq=0 out_seq=0 2024-09-12T03:05:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.579+0000 7f548affd640 1 -- 172.21.15.46:0/640537520 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8af843d70 con 0x55a8af8f2cf0 2024-09-12T03:05:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.579+0000 7f548affd640 10 client.26030 handle_command_reply: tid=0 2024-09-12T03:05:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.579+0000 7f54b4bc9740 1 client.26030 shutdown 2024-09-12T03:05:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 msgr2=0x55a8af8eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096d0 tx=0x7f54a0009f10 comp rx=0 tx=0).stop 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 shutdown_connections 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55a8af8f2cf0 0x55a8af95fdf0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f548affd640 0 client.26030 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8af8e2cb0 0x55a8af8e6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a8af8d8410 0x55a8af8e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 --2- 172.21.15.46:0/640537520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8af8d7af0 0x55a8af8eb2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.580+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 >> 172.21.15.46:0/640537520 conn(0x55a8af8d4060 msgr2=0x55a8af8e1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.581+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 shutdown_connections 2024-09-12T03:05:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.581+0000 7f54b4bc9740 1 -- 172.21.15.46:0/640537520 wait complete. 2024-09-12T03:05:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:05:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:05:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:05:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.581+0000 7f54b4bc9740 20 client.26030 trim_cache size 0 max 16384 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 msgr2=0x7f54940a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 0x7f54940a5440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a409b260 tx=0x7f54a400cf70 comp rx=0 tx=0).stop 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 msgr2=0x7f54ac19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f54a400ee70 tx=0x7f54a400c740 comp rx=0 tx=0).stop 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 shutdown_connections 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f54940a2f80 0x7f54940a5440 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54ac19e690 0x7f54ac19eb10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54ac10d4b0 0x7f54ac1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 --2- 172.21.15.46:0/3514975574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac108360 0x7f54ac1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 >> 172.21.15.46:0/3514975574 conn(0x7f54ac077ad0 msgr2=0x7f54ac078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 shutdown_connections 2024-09-12T03:05:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:30.582+0000 7f54b1e2e640 1 -- 172.21.15.46:0/3514975574 wait complete. 2024-09-12T03:05:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:30 smithi196 ceph-mon[34587]: pgmap v481: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-12T03:05:30.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:30 smithi046 ceph-mon[30951]: pgmap v481: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-12T03:05:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:30 smithi145 ceph-mon[34674]: pgmap v481: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-12T03:05:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:32 smithi196 ceph-mon[34587]: pgmap v482: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-09-12T03:05:32.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:32 smithi046 ceph-mon[30951]: pgmap v482: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-09-12T03:05:32.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:32 smithi145 ceph-mon[34674]: pgmap v482: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.2 MiB/s wr, 300 op/s 2024-09-12T03:05:33.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:33 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:33.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:33 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:33.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:33 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:34 smithi196 ceph-mon[34587]: pgmap v483: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-09-12T03:05:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:34 smithi046 ceph-mon[30951]: pgmap v483: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-09-12T03:05:34.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:34 smithi145 ceph-mon[34674]: pgmap v483: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-09-12T03:05:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:36 smithi196 ceph-mon[34587]: pgmap v484: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1009 KiB/s wr, 256 op/s 2024-09-12T03:05:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:36 smithi046 ceph-mon[30951]: pgmap v484: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1009 KiB/s wr, 256 op/s 2024-09-12T03:05:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:36 smithi145 ceph-mon[34674]: pgmap v484: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1009 KiB/s wr, 256 op/s 2024-09-12T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:38 smithi196 ceph-mon[34587]: pgmap v485: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-09-12T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:38 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:38.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:38 smithi046 ceph-mon[30951]: pgmap v485: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-09-12T03:05:38.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:38 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:38.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:38 smithi145 ceph-mon[34674]: pgmap v485: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.1 MiB/s wr, 293 op/s 2024-09-12T03:05:38.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:38 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:40 smithi196 ceph-mon[34587]: pgmap v486: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-09-12T03:05:40.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:40 smithi046 ceph-mon[30951]: pgmap v486: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-09-12T03:05:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:40 smithi145 ceph-mon[34674]: pgmap v486: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-09-12T03:05:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:42 smithi196 ceph-mon[34587]: pgmap v487: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:05:42.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:42 smithi046 ceph-mon[30951]: pgmap v487: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:05:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:42 smithi145 ceph-mon[34674]: pgmap v487: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 -- 172.21.15.46:0/3063220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 msgr2=0x7f45bc10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 --2- 172.21.15.46:0/3063220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc10e560 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009ca0 tx=0x7f45b402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 -- 172.21.15.46:0/3063220110 shutdown_connections 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 --2- 172.21.15.46:0/3063220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc10e560 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 --2- 172.21.15.46:0/3063220110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 0x7f45bc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 --2- 172.21.15.46:0/3063220110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 -- 172.21.15.46:0/3063220110 >> 172.21.15.46:0/3063220110 conn(0x7f45bc0fc6b0 msgr2=0x7f45bc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.704+0000 7f45c3282640 1 -- 172.21.15.46:0/3063220110 shutdown_connections 2024-09-12T03:05:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.705+0000 7f45c3282640 1 -- 172.21.15.46:0/3063220110 wait complete. 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 Processor -- start 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 -- start start 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 0x7f45bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc111d50 con 0x7f45bc10c110 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc111bd0 con 0x7f45bc102d40 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.706+0000 7f45c3282640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc111ed0 con 0x7f45bc103660 2024-09-12T03:05:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38148/0 (socket says 172.21.15.46:38148) 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 -- 172.21.15.46:0/1796961267 learned_addr learned my addr 172.21.15.46:0/1796961267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c081d640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 0x7f45bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c101e640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 -- 172.21.15.46:0/1796961267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 msgr2=0x7f45bc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 0x7f45bc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 -- 172.21.15.46:0/1796961267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 msgr2=0x7f45bc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c181f640 1 -- 172.21.15.46:0/1796961267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1aac30 con 0x7f45bc10c110 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.707+0000 7f45c101e640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45c181f640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009dc0 tx=0x7f45b4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45b27fc640 1 -- 172.21.15.46:0/1796961267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4003c30 con 0x7f45bc10c110 2024-09-12T03:05:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45b27fc640 1 -- 172.21.15.46:0/1796961267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b4003dd0 con 0x7f45bc10c110 2024-09-12T03:05:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45b27fc640 1 -- 172.21.15.46:0/1796961267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4030aa0 con 0x7f45bc10c110 2024-09-12T03:05:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1aae00 con 0x7f45bc10c110 2024-09-12T03:05:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.708+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1ab210 con 0x7f45bc10c110 2024-09-12T03:05:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.711+0000 7f45b27fc640 1 -- 172.21.15.46:0/1796961267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f45b4030ce0 con 0x7f45bc10c110 2024-09-12T03:05:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.712+0000 7f45b27fc640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 0x7f45a80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.712+0000 7f45b27fc640 1 -- 172.21.15.46:0/1796961267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f45b40f0b60 con 0x7f45bc10c110 2024-09-12T03:05:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c101e640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 0x7f45a80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 Processor -- start 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 -- start start 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a27410 0x55f1b5a31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1b598b2f0 con 0x55f1b5a31cb0 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5933270 con 0x55f1b5a26af0 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c601f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f1b59872b0 con 0x55f1b5a27410 2024-09-12T03:05:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c101e640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 0x7f45a80a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009b50 tx=0x7f45b8005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:60984/0 (socket says 172.21.15.46:60984) 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c101e640 1 -- 172.21.15.46:0/2182872088 learned_addr learned my addr 172.21.15.46:0/2182872088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c181f640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.713+0000 7f45c081d640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a27410 0x55f1b5a31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 -- 172.21.15.46:0/2182872088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a27410 msgr2=0x55f1b5a31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a27410 0x55f1b5a31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 -- 172.21.15.46:0/2182872088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 msgr2=0x55f1b5a340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 -- 172.21.15.46:0/2182872088 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b5932340 con 0x55f1b5a26af0 2024-09-12T03:05:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c181f640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45c101e640 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45b8010ec0 tx=0x7f45b8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c300559b8a105c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45a77fe640 1 -- 172.21.15.46:0/2182872088 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8023020 con 0x55f1b5a26af0 2024-09-12T03:05:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45a77fe640 1 -- 172.21.15.46:0/2182872088 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8004230 con 0x55f1b5a26af0 2024-09-12T03:05:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.714+0000 7f45a77fe640 1 -- 172.21.15.46:0/2182872088 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b801f670 con 0x55f1b5a26af0 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 -- 172.21.15.46:0/2182872088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 msgr2=0x55f1b5a26ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45b8010ec0 tx=0x7f45b8013000 comp rx=0 tx=0).stop 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 -- 172.21.15.46:0/2182872088 shutdown_connections 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a27410 0x55f1b5a31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 --2- 172.21.15.46:0/2182872088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a26ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 -- 172.21.15.46:0/2182872088 >> 172.21.15.46:0/2182872088 conn(0x55f1b5a23060 msgr2=0x55f1b5a23440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 -- 172.21.15.46:0/2182872088 shutdown_connections 2024-09-12T03:05:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.715+0000 7f45c601f740 1 -- 172.21.15.46:0/2182872088 wait complete. 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.716+0000 7f45c601f740 1 -- start start 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.716+0000 7f45c601f740 1 Processor -- start 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.716+0000 7f45c601f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.716+0000 7f45c601f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a2f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c601f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c601f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5a34880 con 0x55f1b5a27410 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c601f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5914290 con 0x55f1b5a26af0 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c601f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5a34a00 con 0x55f1b5a31cb0 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a2f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:60990/0 (socket says 172.21.15.46:60990) 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c101e640 1 -- 172.21.15.46:0/2355722911 learned_addr learned my addr 172.21.15.46:0/2355722911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c601f740 10 client.0 _ll_register_callbacks cb 0x55f1b5945140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c181f640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a2f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 -- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 msgr2=0x55f1b5a2f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a2f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 -- 172.21.15.46:0/2355722911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 msgr2=0x55f1b5a2ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 -- 172.21.15.46:0/2355722911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b569e210 con 0x55f1b5a27410 2024-09-12T03:05:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.717+0000 7f45c081d640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a2f140 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f45bc102150 tx=0x7f45ac012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01564ca18c5e652 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac004030 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f45ac0041d0 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b8004230 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f45ac0043d0 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45c601f740 10 client.16440 fetch_fsmap learned FSMap version 43 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f1b576a620 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8004230 con 0x55f1b5a27410 2024-09-12T03:05:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.718+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f45ac003bd0 con 0x55f1b5a27410 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 10 client.16440 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 20 client.16440 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5a440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 4 client.16440 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b5998a20 con 0x55f1b5a41c90 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.719+0000 7f45c101e640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5a440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.720+0000 7f45c101e640 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5a440e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6679ae6be36f1eee server_cookie=def281478a44abfe in_seq=0 out_seq=0 2024-09-12T03:05:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.720+0000 7f45a4ff9640 10 client.16440 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45a4ff9640 1 -- 172.21.15.46:0/2355722911 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1b5998a20 con 0x55f1b5a41c90 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45a4ff9640 10 client.16440 handle_command_reply: tid=0 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 client.16440 shutdown 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 msgr2=0x55f1b5a2f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a2f140 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f45bc102150 tx=0x7f45ac012b60 comp rx=0 tx=0).stop 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 shutdown_connections 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5a440e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45a4ff9640 0 client.16440 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a2f140 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 --2- 172.21.15.46:0/2355722911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.721+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 >> 172.21.15.46:0/2355722911 conn(0x55f1b5a23060 msgr2=0x55f1b5a221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.722+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 shutdown_connections 2024-09-12T03:05:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.722+0000 7f45c601f740 1 -- 172.21.15.46:0/2355722911 wait complete. 2024-09-12T03:05:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.722+0000 7f45c601f740 20 client.16440 trim_cache size 0 max 16384 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.743+0000 7f45c601f740 1 Processor -- start 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.743+0000 7f45c601f740 1 -- start start 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a49f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c081d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c101e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:32774/0 (socket says 172.21.15.46:32774) 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c101e640 1 -- 172.21.15.46:0/3414288713 learned_addr learned my addr 172.21.15.46:0/3414288713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b7db0 con 0x55f1b5a27410 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b5680 con 0x55f1b5a26af0 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b34f0 con 0x55f1b5a31cb0 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.744+0000 7f45c181f640 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a49f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 -- 172.21.15.46:0/3414288713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 msgr2=0x55f1b5a2f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 -- 172.21.15.46:0/3414288713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 msgr2=0x55f1b5a499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 -- 172.21.15.46:0/3414288713 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b5a24110 con 0x55f1b5a31cb0 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c081d640 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45c181f640 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a49f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45b40edfb0 tx=0x7f45b40bc8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d2cd7450146353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45a6ffd640 1 -- 172.21.15.46:0/3414288713 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bf030 con 0x55f1b5a31cb0 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45a6ffd640 1 -- 172.21.15.46:0/3414288713 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b4036040 con 0x55f1b5a31cb0 2024-09-12T03:05:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.745+0000 7f45a6ffd640 1 -- 172.21.15.46:0/3414288713 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4004680 con 0x55f1b5a31cb0 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 msgr2=0x55f1b5a49f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a49f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45b40edfb0 tx=0x7f45b40bc8b0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 shutdown_connections 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a49f00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a27410 0x55f1b5a499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 --2- 172.21.15.46:0/3414288713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a26af0 0x55f1b5a2f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 >> 172.21.15.46:0/3414288713 conn(0x55f1b5a23060 msgr2=0x55f1b5a30c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 shutdown_connections 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.746+0000 7f45c601f740 1 -- 172.21.15.46:0/3414288713 wait complete. 2024-09-12T03:05:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 -- start start 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 Processor -- start 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a27410 0x55f1b5a360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b3b40 con 0x55f1b5a31cb0 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b4fb0 con 0x55f1b5a27410 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.747+0000 7f45c601f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b4210 con 0x55f1b5a26af0 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c601f740 10 client.0 _ll_register_callbacks cb 0x7f45ac009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:52064/0 (socket says 172.21.15.46:52064) 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 -- 172.21.15.46:0/1592513290 learned_addr learned my addr 172.21.15.46:0/1592513290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c181f640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c081d640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a27410 0x55f1b5a360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 -- 172.21.15.46:0/1592513290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a27410 msgr2=0x55f1b5a360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a27410 0x55f1b5a360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 -- 172.21.15.46:0/1592513290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 msgr2=0x55f1b5a36600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a36600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 -- 172.21.15.46:0/1592513290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b5aabb30 con 0x55f1b5a26af0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45c101e640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45b8010e90 tx=0x7f45b8005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ec67d8e37c771b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.748+0000 7f45a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8016070 con 0x55f1b5a26af0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7f45b8003a20 con 0x55f1b5a26af0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b4036040 con 0x55f1b5a26af0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f45b8044050 con 0x55f1b5a26af0 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 10 client.25999 fetch_fsmap learned FSMap version 43 2024-09-12T03:05:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f1b5aace80 con 0x55f1b5a26af0 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b4036040 con 0x55f1b5a26af0 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f45b803d060 con 0x55f1b5a26af0 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 10 client.25999 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 20 client.25999 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.749+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5aaf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.750+0000 7f45c601f740 4 client.25999 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.750+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1b5992d70 con 0x55f1b5a41c90 2024-09-12T03:05:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.750+0000 7f45c081d640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5aaf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.751+0000 7f45c081d640 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5aaf4d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f8da683bb66b2d server_cookie=60f02301e7558e9c in_seq=0 out_seq=0 2024-09-12T03:05:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.751+0000 7f45a4ff9640 10 client.25999 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45a4ff9640 1 -- 172.21.15.46:0/1592513290 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1b5992d70 con 0x55f1b5a41c90 2024-09-12T03:05:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45a4ff9640 10 client.25999 handle_command_reply: tid=0 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 client.25999 shutdown 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 msgr2=0x55f1b5a35b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45b8010e90 tx=0x7f45b8005bf0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 shutdown_connections 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x55f1b5a41c90 0x55f1b5aaf4d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1b5a31cb0 0x55f1b5a36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45a4ff9640 0 client.25999 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f1b5a27410 0x55f1b5a360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 --2- 172.21.15.46:0/1592513290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1b5a26af0 0x55f1b5a35b80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 >> 172.21.15.46:0/1592513290 conn(0x55f1b5a23060 msgr2=0x55f1b5a28550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 shutdown_connections 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 1 -- 172.21.15.46:0/1592513290 wait complete. 2024-09-12T03:05:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.761+0000 7f45c601f740 20 client.25999 trim_cache size 0 max 16384 2024-09-12T03:05:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:05:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:05:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:05:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 msgr2=0x7f45a80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 0x7f45a80a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009b50 tx=0x7f45b8005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 msgr2=0x7f45bc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009dc0 tx=0x7f45b4032ff0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 shutdown_connections 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45a80a2e20 0x7f45a80a52e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc10c110 0x7f45bc101410 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc103660 0x7f45bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 --2- 172.21.15.46:0/1796961267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45bc102d40 0x7f45bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 >> 172.21.15.46:0/1796961267 conn(0x7f45bc0fc6b0 msgr2=0x7f45bc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 shutdown_connections 2024-09-12T03:05:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:43.762+0000 7f45c3282640 1 -- 172.21.15.46:0/1796961267 wait complete. 2024-09-12T03:05:44.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:44 smithi196 ceph-mon[34587]: pgmap v488: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-09-12T03:05:44.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:44 smithi046 ceph-mon[30951]: pgmap v488: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-09-12T03:05:44.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:44 smithi145 ceph-mon[34674]: pgmap v488: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-09-12T03:05:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:46 smithi196 ceph-mon[34587]: pgmap v489: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 993 KiB/s wr, 253 op/s 2024-09-12T03:05:46.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:46 smithi046 ceph-mon[30951]: pgmap v489: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 993 KiB/s wr, 253 op/s 2024-09-12T03:05:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:46 smithi145 ceph-mon[34674]: pgmap v489: 129 pgs: 129 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 993 KiB/s wr, 253 op/s 2024-09-12T03:05:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:48 smithi196 ceph-mon[34587]: pgmap v490: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-09-12T03:05:48.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:48 smithi046 ceph-mon[30951]: pgmap v490: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-09-12T03:05:48.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:48 smithi145 ceph-mon[34674]: pgmap v490: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-09-12T03:05:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:50 smithi196 ceph-mon[34587]: pgmap v491: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 317 op/s 2024-09-12T03:05:50.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:50 smithi046 ceph-mon[30951]: pgmap v491: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 317 op/s 2024-09-12T03:05:50.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:50 smithi145 ceph-mon[34674]: pgmap v491: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 317 op/s 2024-09-12T03:05:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:52 smithi196 ceph-mon[34587]: pgmap v492: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1022 KiB/s wr, 260 op/s 2024-09-12T03:05:52.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:52 smithi046 ceph-mon[30951]: pgmap v492: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1022 KiB/s wr, 260 op/s 2024-09-12T03:05:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:52 smithi145 ceph-mon[34674]: pgmap v492: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1022 KiB/s wr, 260 op/s 2024-09-12T03:05:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:54 smithi196 ceph-mon[34587]: pgmap v493: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-09-12T03:05:54.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:54 smithi046 ceph-mon[30951]: pgmap v493: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-09-12T03:05:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:54 smithi145 ceph-mon[34674]: pgmap v493: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-09-12T03:05:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:56 smithi196 ceph-mon[34587]: pgmap v494: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 967 KiB/s wr, 246 op/s 2024-09-12T03:05:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:56 smithi046 ceph-mon[30951]: pgmap v494: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 967 KiB/s wr, 246 op/s 2024-09-12T03:05:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:56 smithi145 ceph-mon[34674]: pgmap v494: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 967 KiB/s wr, 246 op/s 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/2743169606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c41035e0 msgr2=0x7fa3c410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/2743169606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c41035e0 0x7fa3c410bac0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402a480 comp rx=0 tx=0).stop 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/2743169606 shutdown_connections 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/2743169606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c410c110 0x7fa3c410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/2743169606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c41035e0 0x7fa3c410bac0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/2743169606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.864+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/2743169606 >> 172.21.15.46:0/2743169606 conn(0x7fa3c40fc6b0 msgr2=0x7fa3c40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/2743169606 shutdown_connections 2024-09-12T03:05:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/2743169606 wait complete. 2024-09-12T03:05:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 Processor -- start 2024-09-12T03:05:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- start start 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c41035e0 0x7fa3c4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4111da0 con 0x7fa3c410c110 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4111c20 con 0x7fa3c41035e0 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3cbfa1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4111f20 con 0x7fa3c4102cc0 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.865+0000 7fa3ca53e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52370/0 (socket says 172.21.15.46:52370) 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 -- 172.21.15.46:0/952667607 learned_addr learned my addr 172.21.15.46:0/952667607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3c953c640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c41035e0 0x7fa3c4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 -- 172.21.15.46:0/952667607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 msgr2=0x7fa3c41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 -- 172.21.15.46:0/952667607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c41035e0 msgr2=0x7fa3c4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c41035e0 0x7fa3c4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 -- 172.21.15.46:0/952667607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419d2e0 con 0x7fa3c410c110 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3ca53e640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00eaa0 tx=0x7fa3bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3baffd640 1 -- 172.21.15.46:0/952667607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00cdb0 con 0x7fa3c410c110 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3baffd640 1 -- 172.21.15.46:0/952667607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3bc005020 con 0x7fa3c410c110 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419d5d0 con 0x7fa3c410c110 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3baffd640 1 -- 172.21.15.46:0/952667607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0116d0 con 0x7fa3c410c110 2024-09-12T03:05:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.866+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a4e60 con 0x7fa3c410c110 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.869+0000 7fa3cee14740 1 Processor -- start 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.869+0000 7fa3baffd640 1 -- 172.21.15.46:0/952667607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa3bc0051d0 con 0x7fa3c410c110 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 -- start start 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee33ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56534ed982f0 con 0x56534ee34410 2024-09-12T03:05:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56534ed40270 con 0x56534ee33af0 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3cee14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56534ed942b0 con 0x56534ee3ecb0 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52384/0 (socket says 172.21.15.46:52384) 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/1186817830 learned_addr learned my addr 172.21.15.46:0/1186817830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c953c640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c953c640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee33ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3baffd640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 0x7fa3b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/1186817830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 msgr2=0x56534ee410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.870+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/1186817830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 msgr2=0x56534ee33ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee33ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/1186817830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534ed3f340 con 0x56534ee34410 2024-09-12T03:05:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3c953c640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee33ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3ca53e640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 0x7fa3b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000bc20 tx=0x7fa3c000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abebb6b050a1de0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3baffd640 1 -- 172.21.15.46:0/952667607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7fa3bc0cfa20 con 0x7fa3c410c110 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3affff640 1 -- 172.21.15.46:0/1186817830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0017620 con 0x56534ee34410 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3affff640 1 -- 172.21.15.46:0/1186817830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3c0017c70 con 0x56534ee34410 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.871+0000 7fa3affff640 1 -- 172.21.15.46:0/1186817830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000fc50 con 0x56534ee34410 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- 172.21.15.46:0/1186817830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 msgr2=0x56534ee3e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3ca53e640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 0x7fa3b00a5260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00efd0 tx=0x7fa3bc086040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000bc20 tx=0x7fa3c000d710 comp rx=0 tx=0).stop 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- 172.21.15.46:0/1186817830 shutdown_connections 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee3e770 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 --2- 172.21.15.46:0/1186817830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee33ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- 172.21.15.46:0/1186817830 >> 172.21.15.46:0/1186817830 conn(0x56534ee30060 msgr2=0x56534ee30440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- 172.21.15.46:0/1186817830 shutdown_connections 2024-09-12T03:05:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- 172.21.15.46:0/1186817830 wait complete. 2024-09-12T03:05:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 -- start start 2024-09-12T03:05:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.872+0000 7fa3cee14740 1 Processor -- start 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee33af0 0x56534ee3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee34410 0x56534ee3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56534ee41880 con 0x56534ee3ecb0 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56534ed21290 con 0x56534ee34410 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56534ee41a00 con 0x56534ee33af0 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3cee14740 10 client.0 _ll_register_callbacks cb 0x56534ed52140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52400/0 (socket says 172.21.15.46:52400) 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 -- 172.21.15.46:0/2294274578 learned_addr learned my addr 172.21.15.46:0/2294274578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee34410 0x56534ee3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee33af0 0x56534ee3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 -- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee33af0 msgr2=0x56534ee3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee33af0 0x56534ee3bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 -- 172.21.15.46:0/2294274578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee34410 msgr2=0x56534ee3c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee34410 0x56534ee3c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.873+0000 7fa3c953c640 1 -- 172.21.15.46:0/2294274578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534eaab210 con 0x56534ee3ecb0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005940 tx=0x7fa3b402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98d28307e61cec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4035070 con 0x56534ee3ecb0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7fa3b4004450 con 0x56534ee3ecb0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c0017c70 con 0x56534ee3ecb0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa3b40035f0 con 0x56534ee3ecb0 2024-09-12T03:05:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 10 client.16458 fetch_fsmap learned FSMap version 43 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56534eb77620 con 0x56534ee3ecb0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3c0017c70 con 0x56534ee3ecb0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa3b403d7f0 con 0x56534ee3ecb0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 10 client.16458 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 20 client.16458 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534ee4eaa0 0x56534ee50e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 4 client.16458 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.874+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534eda5a20 con 0x56534ee4eaa0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.875+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534ee4eaa0 0x56534ee50e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.875+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534ee4eaa0 0x56534ee50e90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759c36deef3dba58 server_cookie=49ad3a90f10c7316 in_seq=0 out_seq=0 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.875+0000 7fa3ad7fa640 10 client.16458 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/2294274578 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56534eda5a20 con 0x56534ee4eaa0 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3ad7fa640 10 client.16458 handle_command_reply: tid=0 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 client.16458 shutdown 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 msgr2=0x56534ee3c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005940 tx=0x7fa3b402a480 comp rx=0 tx=0).stop 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 shutdown_connections 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534ee4eaa0 0x56534ee50e90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee3c5b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee34410 0x56534ee3c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3ad7fa640 0 client.16458 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2294274578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee33af0 0x56534ee3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 >> 172.21.15.46:0/2294274578 conn(0x56534ee30060 msgr2=0x56534ee35dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 shutdown_connections 2024-09-12T03:05:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.876+0000 7fa3cee14740 1 -- 172.21.15.46:0/2294274578 wait complete. 2024-09-12T03:05:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.877+0000 7fa3cee14740 20 client.16458 trim_cache size 0 max 16384 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 Processor -- start 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 -- start start 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee34410 0x56534ee56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41b1780 con 0x56534ee3ecb0 2024-09-12T03:05:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41af050 con 0x56534ee33af0 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3cee14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41acec0 con 0x56534ee34410 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52406/0 (socket says 172.21.15.46:52406) 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 -- 172.21.15.46:0/2485735235 learned_addr learned my addr 172.21.15.46:0/2485735235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 -- 172.21.15.46:0/2485735235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee34410 msgr2=0x56534ee56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee34410 0x56534ee56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 -- 172.21.15.46:0/2485735235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 msgr2=0x56534ee3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee3c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.892+0000 7fa3c953c640 1 -- 172.21.15.46:0/2485735235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534ee3e680 con 0x56534ee3ecb0 2024-09-12T03:05:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3c953c640 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4031440 tx=0x7fa3b40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e7a4b2563e5e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3af7fe640 1 -- 172.21.15.46:0/2485735235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4035070 con 0x56534ee3ecb0 2024-09-12T03:05:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3af7fe640 1 -- 172.21.15.46:0/2485735235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b4004450 con 0x56534ee3ecb0 2024-09-12T03:05:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3af7fe640 1 -- 172.21.15.46:0/2485735235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4005030 con 0x56534ee3ecb0 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 -- 172.21.15.46:0/2485735235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 msgr2=0x56534ee56c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4031440 tx=0x7fa3b40058d0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 -- 172.21.15.46:0/2485735235 shutdown_connections 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee3ecb0 0x56534ee56c70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee34410 0x56534ee56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 --2- 172.21.15.46:0/2485735235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 -- 172.21.15.46:0/2485735235 >> 172.21.15.46:0/2485735235 conn(0x56534ee30060 msgr2=0x56534ee3dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.893+0000 7fa3cee14740 1 -- 172.21.15.46:0/2485735235 shutdown_connections 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 -- 172.21.15.46:0/2485735235 wait complete. 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 -- start start 2024-09-12T03:05:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 Processor -- start 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee42bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ad510 con 0x56534ee34410 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ae980 con 0x56534ee33af0 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41adbe0 con 0x56534ee3ecb0 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3cee14740 10 client.0 _ll_register_callbacks cb 0x7fa3b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.894+0000 7fa3c9d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52422/0 (socket says 172.21.15.46:52422) 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/3484315835 learned_addr learned my addr 172.21.15.46:0/3484315835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c953c640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 msgr2=0x56534ee42bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c953c640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee42bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee42bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/3484315835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 msgr2=0x56534ee47350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee47350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 -- 172.21.15.46:0/3484315835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534ee433c0 con 0x56534ee34410 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c953c640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee42bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3c9d3d640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x56534ee30ec0 tx=0x7fa3c000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d05f842f1d244d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f880 con 0x56534ee34410 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b4004450 con 0x56534ee34410 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9515+0+0 (secure 0 0 0) 0x7fa3c001d040 con 0x56534ee34410 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa3c0038050 con 0x56534ee34410 2024-09-12T03:05:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3cee14740 10 client.16464 fetch_fsmap learned FSMap version 43 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.895+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56534eeb9680 con 0x56534ee34410 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b4004450 con 0x56534ee34410 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa3c0004660 con 0x56534ee34410 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 10 client.16464 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 20 client.16464 populate_metadata read hostname 'smithi046' 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534eeb9bd0 0x56534eebbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 4 client.16464 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 --> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56534ed9fd70 con 0x56534eeb9bd0 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3c953c640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534eeb9bd0 0x56534eebbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3c953c640 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534eeb9bd0 0x56534eebbff0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7059be486686a99 server_cookie=df20fb066002a7a in_seq=0 out_seq=0 2024-09-12T03:05:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.896+0000 7fa3ad7fa640 10 client.16464 ms_handle_connect on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3ad7fa640 1 -- 172.21.15.46:0/3484315835 <== mds.0 v2:172.21.15.196:6838/1262634008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56534ed9fd70 con 0x56534eeb9bd0 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3ad7fa640 10 client.16464 handle_command_reply: tid=0 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 client.16464 shutdown 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 msgr2=0x56534ee42670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x56534ee30ec0 tx=0x7fa3c000f770 comp rx=0 tx=0).stop 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 shutdown_connections 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:6838/1262634008,v1:172.21.15.196:6839/1262634008] conn(0x56534eeb9bd0 0x56534eebbff0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534ee3ecb0 0x56534ee42bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56534ee34410 0x56534ee42670 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 --2- 172.21.15.46:0/3484315835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56534ee33af0 0x56534ee47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3ad7fa640 0 client.16464 ms_handle_reset on v2:172.21.15.196:6838/1262634008 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 >> 172.21.15.46:0/3484315835 conn(0x56534ee30060 msgr2=0x56534ee3d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 shutdown_connections 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 1 -- 172.21.15.46:0/3484315835 wait complete. 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.901+0000 7fa3cee14740 20 client.16464 trim_cache size 0 max 16384 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-09-12T03:05:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 msgr2=0x7fa3b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 0x7fa3b00a5260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00efd0 tx=0x7fa3bc086040 comp rx=0 tx=0).stop 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 msgr2=0x7fa3c419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00eaa0 tx=0x7fa3bc00ef60 comp rx=0 tx=0).stop 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 shutdown_connections 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa3b00a2da0 0x7fa3b00a5260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c410c110 0x7fa3c419cbb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3c41035e0 0x7fa3c4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 --2- 172.21.15.46:0/952667607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102cc0 0x7fa3c41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 >> 172.21.15.46:0/952667607 conn(0x7fa3c40fc6b0 msgr2=0x7fa3c40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 shutdown_connections 2024-09-12T03:05:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-09-12T03:05:56.902+0000 7fa3cbfa1640 1 -- 172.21.15.46:0/952667607 wait complete. 2024-09-12T03:05:57.299 INFO:tasks.workunit.client.0.smithi046.stdout:Syncing()...1 sec 2024-09-12T03:05:57.299 INFO:tasks.workunit.client.0.smithi046.stdout:FFSB benchmark finished at: Thu Sep 12 03:05:57 2024 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:Results: 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:Benchmark took 382.28 sec 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:Total Results 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:=============== 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: ======= ============ ========= ======= =========== ========== 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: write : 1200640 3140.74 100.000% 100.000% 12.3MB/sec 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:- 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout:3140.74 Transactions per Second 2024-09-12T03:05:57.300 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.301 INFO:tasks.workunit.client.0.smithi046.stdout:Throughput Results 2024-09-12T03:05:57.301 INFO:tasks.workunit.client.0.smithi046.stdout:=================== 2024-09-12T03:05:57.301 INFO:tasks.workunit.client.0.smithi046.stdout:Write Throughput: 12.3MB/sec 2024-09-12T03:05:57.305 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.305 INFO:tasks.workunit.client.0.smithi046.stdout:System Call Latency statistics in millisecs 2024-09-12T03:05:57.305 INFO:tasks.workunit.client.0.smithi046.stdout:===== 2024-09-12T03:05:57.305 INFO:tasks.workunit.client.0.smithi046.stdout: Min Avg Max Total Calls 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: ======== ======== ======== ============ 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout:[ open] 0.462000 617.070988 22217.796875 938 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[0] 5000.000000 - 10000.000000 : 17 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[1] 2000.000000 - 5000.000000 : 43 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[2] 1000.000000 - 2000.000000 : 23 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[3] 500.000000 - 1000.000000 : 8 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[4] 200.000000 - 500.000000 : 15 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[5] 100.000000 - 200.000000 : 23 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[6] 50.000000 - 100.000000 : 173 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[7] 20.000000 - 50.000000 : 380 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[8] 10.000000 - 20.000000 : 139 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[9] 5.000000 - 10.000000 : 46 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[10] 2.000000 - 5.000000 : 25 2024-09-12T03:05:57.306 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[11] 1.000000 - 2.000000 : 25 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[12] 0.500000 - 1.000000 : 3 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout:[ write] 0.096000 8.541834 40941.066406 1200640 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[0] 5000.000000 - 10000.000000 : 75 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[1] 2000.000000 - 5000.000000 : 48 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[2] 1000.000000 - 2000.000000 : 29 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[3] 500.000000 - 1000.000000 : 173 2024-09-12T03:05:57.307 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[4] 200.000000 - 500.000000 : 917 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[5] 100.000000 - 200.000000 : 3159 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[6] 50.000000 - 100.000000 : 26652 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[7] 20.000000 - 50.000000 : 80084 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[8] 10.000000 - 20.000000 : 105766 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[9] 5.000000 - 10.000000 : 124533 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[10] 2.000000 - 5.000000 : 115982 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[11] 1.000000 - 2.000000 : 47398 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[12] 0.500000 - 1.000000 : 57068 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[13] 0.200000 - 0.500000 : 301441 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[14] 0.100000 - 0.200000 : 337186 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[15] 0.050000 - 0.100000 : 43 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-12T03:05:57.308 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout:[ lseek] 0.000000 0.001145 6.252000 1200640 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-12T03:05:57.309 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[9] 5.000000 - 10.000000 : 1 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[11] 1.000000 - 2.000000 : 3 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[12] 0.500000 - 1.000000 : 2 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[13] 0.200000 - 0.500000 : 12 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[14] 0.100000 - 0.200000 : 19 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[15] 0.050000 - 0.100000 : 16 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[16] 0.020000 - 0.050000 : 262 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[17] 0.010000 - 0.020000 : 1265 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[18] 0.000000 - 0.010000 : 1199058 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout:[ close] 0.026000 2.884016 108.999001 938 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T03:05:57.310 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[6] 50.000000 - 100.000000 : 7 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[7] 20.000000 - 50.000000 : 30 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[8] 10.000000 - 20.000000 : 34 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[9] 5.000000 - 10.000000 : 44 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[10] 2.000000 - 5.000000 : 54 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[11] 1.000000 - 2.000000 : 26 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[12] 0.500000 - 1.000000 : 30 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[13] 0.200000 - 0.500000 : 63 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[14] 0.100000 - 0.200000 : 153 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[15] 0.050000 - 0.100000 : 64 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[16] 0.020000 - 0.050000 : 431 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-12T03:05:57.311 INFO:tasks.workunit.client.0.smithi046.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout:0.6% User Time 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout:33.7% System Time 2024-09-12T03:05:57.312 INFO:tasks.workunit.client.0.smithi046.stdout:34.3% CPU Utilization 2024-09-12T03:05:57.317 INFO:tasks.workunit.client.0.smithi046.stderr:+ cd .. 2024-09-12T03:05:57.323 INFO:tasks.workunit.client.0.smithi046.stderr:+ rm -r tmp ffsb 2024-09-12T03:05:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:58 smithi196 ceph-mon[34587]: pgmap v495: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1020 KiB/s wr, 260 op/s 2024-09-12T03:05:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:58 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:58 smithi046 ceph-mon[30951]: pgmap v495: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1020 KiB/s wr, 260 op/s 2024-09-12T03:05:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:58 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:58 smithi145 ceph-mon[34674]: pgmap v495: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1020 KiB/s wr, 260 op/s 2024-09-12T03:05:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:58 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:59.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:59 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:59.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:59 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:59.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:05:59 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:05:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:59 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:59 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:05:59 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:05:59.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:59 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:59.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:59 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:59.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:05:59 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:06:00.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:00 smithi046 ceph-mon[30951]: pgmap v496: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-09-12T03:06:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:00 smithi145 ceph-mon[34674]: pgmap v496: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-09-12T03:06:00.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:00 smithi196 ceph-mon[34587]: pgmap v496: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-09-12T03:06:02.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:02 smithi046 ceph-mon[30951]: pgmap v497: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 759 KiB/s wr, 196 op/s 2024-09-12T03:06:02.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:02 smithi145 ceph-mon[34674]: pgmap v497: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 759 KiB/s wr, 196 op/s 2024-09-12T03:06:02.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:02 smithi196 ceph-mon[34587]: pgmap v497: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 759 KiB/s wr, 196 op/s 2024-09-12T03:06:03.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:03 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:03.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:03 smithi046 ceph-mon[30951]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:06:03.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:03 smithi046 ceph-mon[30951]: pgmap v499: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 565 KiB/s wr, 105 op/s 2024-09-12T03:06:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:03 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:03 smithi145 ceph-mon[34674]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:06:03.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:03 smithi145 ceph-mon[34674]: pgmap v499: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 565 KiB/s wr, 105 op/s 2024-09-12T03:06:03.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:03 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:03.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:03 smithi196 ceph-mon[34587]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:06:03.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:03 smithi196 ceph-mon[34587]: pgmap v499: 129 pgs: 129 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 565 KiB/s wr, 105 op/s 2024-09-12T03:06:06.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:05 smithi145 ceph-mon[34674]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 544 KiB/s wr, 103 op/s 2024-09-12T03:06:06.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:05 smithi196 ceph-mon[34587]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 544 KiB/s wr, 103 op/s 2024-09-12T03:06:06.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:05 smithi046 ceph-mon[30951]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 544 KiB/s wr, 103 op/s 2024-09-12T03:06:07.396 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:06:07.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:06:07.696 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-12T03:06:07.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:06:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:07 smithi046 ceph-mon[30951]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 510 KiB/s wr, 111 op/s 2024-09-12T03:06:08.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:07 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:07 smithi145 ceph-mon[34674]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 510 KiB/s wr, 111 op/s 2024-09-12T03:06:08.110 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:07 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:08.218 DEBUG:teuthology.parallel:result is None 2024-09-12T03:06:08.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:06:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:07 smithi196 ceph-mon[34587]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 510 KiB/s wr, 111 op/s 2024-09-12T03:06:08.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:07 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:08.302 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:06:08.303 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T03:06:08.315 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T03:06:08.326 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T03:06:08.411 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T03:06:08.411 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:06:08.411 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T03:06:08.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T03:06:08.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T03:06:08.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:06:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.769+0000 7f518c93a640 1 -- 172.21.15.46:0/809150729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 msgr2=0x7f51800b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:08.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.769+0000 7f518c93a640 1 --2- 172.21.15.46:0/809150729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b9900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5188050280 tx=0x7f518808e350 comp rx=0 tx=0).stop 2024-09-12T03:06:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.775+0000 7f518c93a640 1 -- 172.21.15.46:0/809150729 shutdown_connections 2024-09-12T03:06:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.775+0000 7f518c93a640 1 --2- 172.21.15.46:0/809150729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b9900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.775+0000 7f518c93a640 1 --2- 172.21.15.46:0/809150729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.775+0000 7f518c93a640 1 --2- 172.21.15.46:0/809150729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51800a47f0 0x7f51800a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.776+0000 7f518c93a640 1 -- 172.21.15.46:0/809150729 >> 172.21.15.46:0/809150729 conn(0x7f518001a0d0 msgr2=0x7f518001a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.783+0000 7f518c93a640 1 -- 172.21.15.46:0/809150729 shutdown_connections 2024-09-12T03:06:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.784+0000 7f518c93a640 1 -- 172.21.15.46:0/809150729 wait complete. 2024-09-12T03:06:08.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.785+0000 7f518c93a640 1 Processor -- start 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 -- start start 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51800a47f0 0x7f51800b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51800aefb0 con 0x7f51800a5110 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f51800aee30 con 0x7f51800a47f0 2024-09-12T03:06:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.786+0000 7f518c93a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51800af130 con 0x7f51800b7510 2024-09-12T03:06:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.787+0000 7f518659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.787+0000 7f518659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32926/0 (socket says 172.21.15.46:32926) 2024-09-12T03:06:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.787+0000 7f518659c640 1 -- 172.21.15.46:0/1003995493 learned_addr learned my addr 172.21.15.46:0/1003995493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:08.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.787+0000 7f518759e640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.788+0000 7f518659c640 1 -- 172.21.15.46:0/1003995493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 msgr2=0x7f51800b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.788+0000 7f518659c640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.788+0000 7f518659c640 1 -- 172.21.15.46:0/1003995493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51800a47f0 msgr2=0x7f51800b6ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.788+0000 7f518659c640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51800a47f0 0x7f51800b6ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.788+0000 7f518659c640 1 -- 172.21.15.46:0/1003995493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800b35e0 con 0x7f51800a5110 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.789+0000 7f518659c640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f517c01fa90 tx=0x7f517c01ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:08.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.789+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c01dcb0 con 0x7f51800a5110 2024-09-12T03:06:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.789+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51800b38d0 con 0x7f51800a5110 2024-09-12T03:06:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.789+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51801476d0 con 0x7f51800a5110 2024-09-12T03:06:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.791+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517c01de50 con 0x7f51800a5110 2024-09-12T03:06:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.791+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c021440 con 0x7f51800a5110 2024-09-12T03:06:08.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.792+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51800a5b20 con 0x7f51800a5110 2024-09-12T03:06:08.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.795+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f517c0215e0 con 0x7f51800a5110 2024-09-12T03:06:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.796+0000 7f516ffff640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 0x7f51700a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.797+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7f517c0e1bb0 con 0x7f51800a5110 2024-09-12T03:06:08.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.804+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f517c0ae360 con 0x7f51800a5110 2024-09-12T03:06:08.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.807+0000 7f5186d9d640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 0x7f51700a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:08.820+0000 7f5186d9d640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 0x7f51700a53d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5178009670 tx=0x7f5178006bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.197+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f51800028c0 con 0x7f51700a2f10 2024-09-12T03:06:09.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:08 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.237+0000 7f516ffff640 1 -- 172.21.15.46:0/1003995493 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f51800028c0 con 0x7f51700a2f10 2024-09-12T03:06:09.238 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T02:50:11", "first": "2024-09-12T02:51:00", "last": "2024-09-12T03:06:00", "last_pruned": "2024-09-12T03:06:01", "created_count": 16, "pruned_count": 9, "active": true} 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 msgr2=0x7f51700a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 0x7f51700a53d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5178009670 tx=0x7f5178006bd0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 msgr2=0x7f51800b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f517c01fa90 tx=0x7f517c01ff50 comp rx=0 tx=0).stop 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 shutdown_connections 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f51700a2f10 0x7f51700a53d0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5178009670 tx=0x7f5178006bd0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51800b7510 0x7f51800b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51800a5110 0x7f51800b2b60 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 --2- 172.21.15.46:0/1003995493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51800a47f0 0x7f51800b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 >> 172.21.15.46:0/1003995493 conn(0x7f518001a0d0 msgr2=0x7f51800b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 shutdown_connections 2024-09-12T03:06:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.241+0000 7f518c93a640 1 -- 172.21.15.46:0/1003995493 wait complete. 2024-09-12T03:06:09.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:06:09.258 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:08 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:09.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:08 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.473+0000 7f0e6285e640 1 -- 172.21.15.46:0/2415344258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c108360 msgr2=0x7f0e5c108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.473+0000 7f0e6285e640 1 --2- 172.21.15.46:0/2415344258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c108360 0x7f0e5c108740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009900 tx=0x7f0e4c02a440 comp rx=0 tx=0).stop 2024-09-12T03:06:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.477+0000 7f0e6285e640 1 -- 172.21.15.46:0/2415344258 shutdown_connections 2024-09-12T03:06:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.477+0000 7f0e6285e640 1 --2- 172.21.15.46:0/2415344258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c10d4b0 0x7f0e5c115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.477+0000 7f0e6285e640 1 --2- 172.21.15.46:0/2415344258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c108d10 0x7f0e5c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.477+0000 7f0e6285e640 1 --2- 172.21.15.46:0/2415344258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c108360 0x7f0e5c108740 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.477+0000 7f0e6285e640 1 -- 172.21.15.46:0/2415344258 >> 172.21.15.46:0/2415344258 conn(0x7f0e5c077ad0 msgr2=0x7f0e5c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:09.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.484+0000 7f0e6285e640 1 -- 172.21.15.46:0/2415344258 shutdown_connections 2024-09-12T03:06:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.488+0000 7f0e6285e640 1 -- 172.21.15.46:0/2415344258 wait complete. 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.488+0000 7f0e6285e640 1 Processor -- start 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 -- start start 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c10d4b0 0x7f0e5c06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c074040 0x7f0e5c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1184f0 con 0x7f0e5c10d4b0 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c118370 con 0x7f0e5c108360 2024-09-12T03:06:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.489+0000 7f0e6285e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c118670 con 0x7f0e5c074040 2024-09-12T03:06:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.490+0000 7f0e60dfb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c074040 0x7f0e5c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.492+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.493+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:39718/0 (socket says 172.21.15.46:39718) 2024-09-12T03:06:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.493+0000 7f0e5bfff640 1 -- 172.21.15.46:0/500730669 learned_addr learned my addr 172.21.15.46:0/500730669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.494+0000 7f0e5bfff640 1 -- 172.21.15.46:0/500730669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c074040 msgr2=0x7f0e5c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.494+0000 7f0e5bfff640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c074040 0x7f0e5c06d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.494+0000 7f0e5bfff640 1 -- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c10d4b0 msgr2=0x7f0e5c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.494+0000 7f0e5bfff640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c10d4b0 0x7f0e5c06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.494+0000 7f0e5bfff640 1 -- 172.21.15.46:0/500730669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c071510 con 0x7f0e5c108360 2024-09-12T03:06:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.501+0000 7f0e5bfff640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02a9b0 tx=0x7f0e4c003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.506+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c03f030 con 0x7f0e5c108360 2024-09-12T03:06:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.506+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e4c032bf0 con 0x7f0e5c108360 2024-09-12T03:06:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.506+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c035070 con 0x7f0e5c108360 2024-09-12T03:06:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.507+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c0717a0 con 0x7f0e5c108360 2024-09-12T03:06:09.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.507+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c071c40 con 0x7f0e5c108360 2024-09-12T03:06:09.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.509+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5c109830 con 0x7f0e5c108360 2024-09-12T03:06:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.514+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0e4c042030 con 0x7f0e5c108360 2024-09-12T03:06:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.515+0000 7f0e59ffb640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 0x7f0e480a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:09.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.515+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7f0e4c0f0280 con 0x7f0e5c108360 2024-09-12T03:06:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.520+0000 7f0e53fff640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 0x7f0e480a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.521+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e4c049030 con 0x7f0e5c108360 2024-09-12T03:06:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.525+0000 7f0e53fff640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 0x7f0e480a5490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c06e230 tx=0x7f0e44006c40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:09.914+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 --> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0e5c108d10 con 0x7f0e480a2fd0 2024-09-12T03:06:10.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.112+0000 7f0e59ffb640 1 -- 172.21.15.46:0/500730669 <== mgr.15228 v2:172.21.15.46:6800/1256678118 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0e5c108d10 con 0x7f0e480a2fd0 2024-09-12T03:06:10.113 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-09-12T03:06:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.115+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 msgr2=0x7f0e480a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.115+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 0x7f0e480a5490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c06e230 tx=0x7f0e44006c40 comp rx=0 tx=0).stop 2024-09-12T03:06:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 msgr2=0x7f0e5c0739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02a9b0 tx=0x7f0e4c003e30 comp rx=0 tx=0).stop 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 shutdown_connections 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f0e480a2fd0 0x7f0e480a5490 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5c074040 0x7f0e5c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e5c10d4b0 0x7f0e5c06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 --2- 172.21.15.46:0/500730669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e5c108360 0x7f0e5c0739f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 >> 172.21.15.46:0/500730669 conn(0x7f0e5c077ad0 msgr2=0x7f0e5c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 shutdown_connections 2024-09-12T03:06:10.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.116+0000 7f0e6285e640 1 -- 172.21.15.46:0/500730669 wait complete. 2024-09-12T03:06:10.134 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:09 smithi046 ceph-mon[30951]: pgmap v502: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 107 op/s 2024-09-12T03:06:10.134 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:09 smithi046 ceph-mon[30951]: from='client.16470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:10.135 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:06:10.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:06:10.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:09 smithi196 ceph-mon[34587]: pgmap v502: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 107 op/s 2024-09-12T03:06:10.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:09 smithi196 ceph-mon[34587]: from='client.16470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.348+0000 7fb323ee4640 1 -- 172.21.15.46:0/3664882203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 msgr2=0x7fb31c10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.348+0000 7fb323ee4640 1 --2- 172.21.15.46:0/3664882203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c10d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb314009cd0 tx=0x7fb31402a910 comp rx=0 tx=0).stop 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.351+0000 7fb323ee4640 1 -- 172.21.15.46:0/3664882203 shutdown_connections 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.351+0000 7fb323ee4640 1 --2- 172.21.15.46:0/3664882203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c10d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.351+0000 7fb323ee4640 1 --2- 172.21.15.46:0/3664882203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c1009b0 0x7fb31c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.351+0000 7fb323ee4640 1 --2- 172.21.15.46:0/3664882203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c106ad0 0x7fb31c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.351+0000 7fb323ee4640 1 -- 172.21.15.46:0/3664882203 >> 172.21.15.46:0/3664882203 conn(0x7fb31c0fc6b0 msgr2=0x7fb31c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.356+0000 7fb323ee4640 1 -- 172.21.15.46:0/3664882203 shutdown_connections 2024-09-12T03:06:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.358+0000 7fb323ee4640 1 -- 172.21.15.46:0/3664882203 wait complete. 2024-09-12T03:06:10.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:09 smithi145 ceph-mon[34674]: pgmap v502: 129 pgs: 19 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 63 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 107 op/s 2024-09-12T03:06:10.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:09 smithi145 ceph-mon[34674]: from='client.16470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.360+0000 7fb323ee4640 1 Processor -- start 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.360+0000 7fb323ee4640 1 -- start start 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.360+0000 7fb323ee4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 0x7fb31c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.361+0000 7fb323ee4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.361+0000 7fb323ee4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 0x7fb31c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.361+0000 7fb323ee4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c111eb0 con 0x7fb31c1009b0 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.361+0000 7fb323ee4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c111d30 con 0x7fb31c106ad0 2024-09-12T03:06:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.361+0000 7fb323ee4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c112030 con 0x7fb31c101350 2024-09-12T03:06:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.363+0000 7fb321c80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 0x7fb31c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.363+0000 7fb321c80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 0x7fb31c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32966/0 (socket says 172.21.15.46:32966) 2024-09-12T03:06:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.363+0000 7fb321c80640 1 -- 172.21.15.46:0/1058708643 learned_addr learned my addr 172.21.15.46:0/1058708643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.363+0000 7fb32147f640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb322481640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 0x7fb31c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb32147f640 1 -- 172.21.15.46:0/1058708643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 msgr2=0x7fb31c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb32147f640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 0x7fb31c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb32147f640 1 -- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 msgr2=0x7fb31c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb32147f640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 0x7fb31c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.366+0000 7fb32147f640 1 -- 172.21.15.46:0/1058708643 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c19d3b0 con 0x7fb31c101350 2024-09-12T03:06:10.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.367+0000 7fb322481640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 0x7fb31c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.372+0000 7fb32147f640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c198900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007c60 tx=0x7fb30c010420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:10.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.374+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c01b020 con 0x7fb31c101350 2024-09-12T03:06:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.374+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb30c007e10 con 0x7fb31c101350 2024-09-12T03:06:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.374+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00c8a0 con 0x7fb31c101350 2024-09-12T03:06:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.374+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c19d6a0 con 0x7fb31c101350 2024-09-12T03:06:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.374+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c10e350 con 0x7fb31c101350 2024-09-12T03:06:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.384+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb300005290 con 0x7fb31c101350 2024-09-12T03:06:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.389+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb30c00cb00 con 0x7fb31c101350 2024-09-12T03:06:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.390+0000 7fb312ffd640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 0x7fb3080a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.390+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7fb30c0d2e60 con 0x7fb31c101350 2024-09-12T03:06:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.392+0000 7fb321c80640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 0x7fb3080a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:10.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.398+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb30c025020 con 0x7fb31c101350 2024-09-12T03:06:10.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.401+0000 7fb321c80640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 0x7fb3080a5390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3180058c0 tx=0x7fb31800a380 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.754+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb300005850 con 0x7fb31c101350 2024-09-12T03:06:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.756+0000 7fb312ffd640 1 -- 172.21.15.46:0/1058708643 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+25405 (secure 0 0 0) 0x7fb30c09f6a0 con 0x7fb31c101350 2024-09-12T03:06:10.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T03:06:10.757 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":113,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T03:06:02.478026+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/328833784":"2024-09-13T02:50:10.431130+0000","172.21.15.46:6801/3748867549":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2520496515":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/1022575768":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/3748867549":"2024-09-13T02:50:10.431130+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2008665137":"2024-09-13T02:50:10.431130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.760+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 msgr2=0x7fb3080a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.760+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 0x7fb3080a5390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3180058c0 tx=0x7fb31800a380 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.760+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 msgr2=0x7fb31c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.760+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c198900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007c60 tx=0x7fb30c010420 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 shutdown_connections 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fb3080a2ed0 0x7fb3080a5390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb31c106ad0 0x7fb31c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb31c101350 0x7fb31c198900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 --2- 172.21.15.46:0/1058708643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c1009b0 0x7fb31c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 >> 172.21.15.46:0/1058708643 conn(0x7fb31c0fc6b0 msgr2=0x7fb31c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 shutdown_connections 2024-09-12T03:06:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.761+0000 7fb323ee4640 1 -- 172.21.15.46:0/1058708643 wait complete. 2024-09-12T03:06:10.789 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:42:56.661399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T02:47:37.749578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T02:47:40.084230+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T02:47:46.560164+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T03:06:10.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:06:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.994+0000 7fbfd41c4640 1 -- 172.21.15.46:0/1606822932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc1091f0 msgr2=0x7fbfcc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:10.994+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/1606822932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1095d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ca0 tx=0x7fbfc4032de0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.001+0000 7fbfd41c4640 1 -- 172.21.15.46:0/1606822932 shutdown_connections 2024-09-12T03:06:11.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.001+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/1606822932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc114ab0 0x7fbfcc116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.001+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/1606822932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.001+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/1606822932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1095d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.001+0000 7fbfd41c4640 1 -- 172.21.15.46:0/1606822932 >> 172.21.15.46:0/1606822932 conn(0x7fbfcc06cdd0 msgr2=0x7fbfcc06d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.009+0000 7fbfd41c4640 1 -- 172.21.15.46:0/1606822932 shutdown_connections 2024-09-12T03:06:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.014+0000 7fbfd41c4640 1 -- 172.21.15.46:0/1606822932 wait complete. 2024-09-12T03:06:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.014+0000 7fbfd41c4640 1 Processor -- start 2024-09-12T03:06:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 -- start start 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbfcc114ab0 0x7fbfcc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1184e0 con 0x7fbfcc1091f0 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc118360 con 0x7fbfcc114ab0 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.016+0000 7fbfd41c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc118660 con 0x7fbfcc109ba0 2024-09-12T03:06:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.018+0000 7fbfd175f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.019+0000 7fbfd1f60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.019+0000 7fbfd1f60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32984/0 (socket says 172.21.15.46:32984) 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.020+0000 7fbfd1f60640 1 -- 172.21.15.46:0/2098976299 learned_addr learned my addr 172.21.15.46:0/2098976299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.021+0000 7fbfd175f640 1 -- 172.21.15.46:0/2098976299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbfcc114ab0 msgr2=0x7fbfcc19e890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.021+0000 7fbfd175f640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbfcc114ab0 0x7fbfcc19e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.021+0000 7fbfd175f640 1 -- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 msgr2=0x7fbfcc1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.021+0000 7fbfd175f640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.021+0000 7fbfd175f640 1 -- 172.21.15.46:0/2098976299 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc19efe0 con 0x7fbfcc109ba0 2024-09-12T03:06:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.024+0000 7fbfd175f640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc1a4c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000ea20 tx=0x7fbfc000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.028+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000ccc0 con 0x7fbfcc109ba0 2024-09-12T03:06:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.028+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc19f2d0 con 0x7fbfcc109ba0 2024-09-12T03:06:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.028+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1bc5b0 con 0x7fbfcc109ba0 2024-09-12T03:06:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.029+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfc000ce60 con 0x7fbfcc109ba0 2024-09-12T03:06:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.030+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00187c0 con 0x7fbfcc109ba0 2024-09-12T03:06:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.032+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbfc0018a20 con 0x7fbfcc109ba0 2024-09-12T03:06:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.033+0000 7fbfcaffd640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 0x7fbfb40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.034+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7fbfc00d0a80 con 0x7fbfcc109ba0 2024-09-12T03:06:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.034+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb8005290 con 0x7fbfcc109ba0 2024-09-12T03:06:11.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.038+0000 7fbfd1f60640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 0x7fbfb40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.042+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfc009d230 con 0x7fbfcc109ba0 2024-09-12T03:06:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.044+0000 7fbfd1f60640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 0x7fbfb40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40333d0 tx=0x7fbfc4011040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:10 smithi196 ceph-mon[34587]: from='client.26063 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:11.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:10 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1058708643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:06:11.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:10 smithi046 ceph-mon[30951]: from='client.26063 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:11.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:10 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1058708643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:06:11.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:10 smithi145 ceph-mon[34674]: from='client.26063 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:11.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:10 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1058708643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:06:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.392+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfb8005850 con 0x7fbfcc109ba0 2024-09-12T03:06:11.394 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-12T03:06:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.394+0000 7fbfcaffd640 1 -- 172.21.15.46:0/2098976299 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fbfc009cb30 con 0x7fbfcc109ba0 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 msgr2=0x7fbfb40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 0x7fbfb40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40333d0 tx=0x7fbfc4011040 comp rx=0 tx=0).stop 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 msgr2=0x7fbfcc1a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc1a4c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000ea20 tx=0x7fbfc000eee0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 shutdown_connections 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fbfb40a2f00 0x7fbfb40a53c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbfcc114ab0 0x7fbfcc19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc109ba0 0x7fbfcc1a4c20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 --2- 172.21.15.46:0/2098976299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfcc1091f0 0x7fbfcc1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.398+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 >> 172.21.15.46:0/2098976299 conn(0x7fbfcc06cdd0 msgr2=0x7fbfcc070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.399+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 shutdown_connections 2024-09-12T03:06:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.399+0000 7fbfd41c4640 1 -- 172.21.15.46:0/2098976299 wait complete. 2024-09-12T03:06:11.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.625+0000 7fdd3f59e640 1 -- 172.21.15.46:0/1866418390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380b2e60 msgr2=0x7fdd380b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.625+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/1866418390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380b2e60 0x7fdd380b5290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdd40067ef0 tx=0x7fdd4008e3c0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.626+0000 7fdd3f59e640 1 -- 172.21.15.46:0/1866418390 shutdown_connections 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.626+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/1866418390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380b2e60 0x7fdd380b5290 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.626+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/1866418390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a5380 0x7fdd380b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.626+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/1866418390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380a4a60 0x7fdd380a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.626+0000 7fdd3f59e640 1 -- 172.21.15.46:0/1866418390 >> 172.21.15.46:0/1866418390 conn(0x7fdd3801a0d0 msgr2=0x7fdd3801a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:11.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.630+0000 7fdd3f59e640 1 -- 172.21.15.46:0/1866418390 shutdown_connections 2024-09-12T03:06:11.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.632+0000 7fdd3f59e640 1 -- 172.21.15.46:0/1866418390 wait complete. 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.633+0000 7fdd3f59e640 1 Processor -- start 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.633+0000 7fdd3f59e640 1 -- start start 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.633+0000 7fdd3f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 0x7fdd3813f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.633+0000 7fdd3f59e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380a5380 0x7fdd3813f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.633+0000 7fdd3f59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 0x7fdd38143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3f59e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380b8b00 con 0x7fdd380a4a60 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3f59e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380b8980 con 0x7fdd380a5380 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3f59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380b8c80 con 0x7fdd380b2e60 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 0x7fdd38143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 0x7fdd38143a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:37566/0 (socket says 172.21.15.46:37566) 2024-09-12T03:06:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.634+0000 7fdd3ed9d640 1 -- 172.21.15.46:0/3564542002 learned_addr learned my addr 172.21.15.46:0/3564542002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:11.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.636+0000 7fdd3e59c640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 0x7fdd3813f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.639+0000 7fdd3e59c640 1 -- 172.21.15.46:0/3564542002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 msgr2=0x7fdd38143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.639+0000 7fdd3e59c640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 0x7fdd38143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.639+0000 7fdd3e59c640 1 -- 172.21.15.46:0/3564542002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380a5380 msgr2=0x7fdd3813f690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:06:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.639+0000 7fdd3e59c640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380a5380 0x7fdd3813f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:11.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.639+0000 7fdd3e59c640 1 -- 172.21.15.46:0/3564542002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd38144140 con 0x7fdd380a4a60 2024-09-12T03:06:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.644+0000 7fdd3e59c640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 0x7fdd3813f150 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009fd0 tx=0x7fdd3000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.647+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30015020 con 0x7fdd380a4a60 2024-09-12T03:06:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.647+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd30011ce0 con 0x7fdd380a4a60 2024-09-12T03:06:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.647+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30010e00 con 0x7fdd380a4a60 2024-09-12T03:06:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.649+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd38144430 con 0x7fdd380a4a60 2024-09-12T03:06:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.649+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd38144840 con 0x7fdd380a4a60 2024-09-12T03:06:11.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.650+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd380a4ec0 con 0x7fdd380a4a60 2024-09-12T03:06:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.657+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdd30018020 con 0x7fdd380a4a60 2024-09-12T03:06:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.658+0000 7fdd2f7fe640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 0x7fdd280a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.659+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7fdd30098fa0 con 0x7fdd380a4a60 2024-09-12T03:06:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.663+0000 7fdd3dd9b640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 0x7fdd280a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:11.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.670+0000 7fdd3dd9b640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 0x7fdd280a5340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdd380a7b00 tx=0x7fdd34006bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:11.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:11.672+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd38144840 con 0x7fdd380a4a60 2024-09-12T03:06:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.028+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd38002610 con 0x7fdd380a4a60 2024-09-12T03:06:12.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.028+0000 7fdd2f7fe640 1 -- 172.21.15.46:0/3564542002 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fdd38144840 con 0x7fdd380a4a60 2024-09-12T03:06:12.029 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-12T03:06:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.032+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 msgr2=0x7fdd280a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.032+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 0x7fdd280a5340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdd380a7b00 tx=0x7fdd34006bd0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.032+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 msgr2=0x7fdd3813f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.032+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 0x7fdd3813f150 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009fd0 tx=0x7fdd3000b870 comp rx=0 tx=0).stop 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 shutdown_connections 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdd280a2e80 0x7fdd280a5340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd380b2e60 0x7fdd38143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd380a5380 0x7fdd3813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 --2- 172.21.15.46:0/3564542002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380a4a60 0x7fdd3813f150 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.034+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 >> 172.21.15.46:0/3564542002 conn(0x7fdd3801a0d0 msgr2=0x7fdd380b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.036+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 shutdown_connections 2024-09-12T03:06:12.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.036+0000 7fdd3f59e640 1 -- 172.21.15.46:0/3564542002 wait complete. 2024-09-12T03:06:12.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:06:12.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:11 smithi196 ceph-mon[34587]: pgmap v503: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-12T03:06:12.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:11 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2098976299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:06:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.274+0000 7fcc17cd9640 1 -- 172.21.15.46:0/1015206904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10105e90 msgr2=0x7fcc10106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.274+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/1015206904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10105e90 0x7fcc10106300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009900 tx=0x7fcc0802a480 comp rx=0 tx=0).stop 2024-09-12T03:06:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.281+0000 7fcc17cd9640 1 -- 172.21.15.46:0/1015206904 shutdown_connections 2024-09-12T03:06:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.281+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/1015206904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10105e90 0x7fcc10106300 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.281+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/1015206904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc10101380 0x7fcc10105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.281+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/1015206904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc10100a60 0x7fcc10100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.281+0000 7fcc17cd9640 1 -- 172.21.15.46:0/1015206904 >> 172.21.15.46:0/1015206904 conn(0x7fcc100fc6a0 msgr2=0x7fcc100feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:11 smithi046 ceph-mon[30951]: pgmap v503: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-12T03:06:12.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:11 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2098976299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:06:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.282+0000 7fcc17cd9640 1 -- 172.21.15.46:0/1015206904 shutdown_connections 2024-09-12T03:06:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.282+0000 7fcc17cd9640 1 -- 172.21.15.46:0/1015206904 wait complete. 2024-09-12T03:06:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.283+0000 7fcc17cd9640 1 Processor -- start 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.283+0000 7fcc17cd9640 1 -- start start 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.283+0000 7fcc17cd9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc10100a60 0x7fcc10073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.283+0000 7fcc17cd9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc17cd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc10105e90 0x7fcc1006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc17cd9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10118340 con 0x7fcc10101380 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc17cd9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc101181c0 con 0x7fcc10100a60 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc17cd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc101184c0 con 0x7fcc10105e90 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc15274640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc15274640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33010/0 (socket says 172.21.15.46:33010) 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.284+0000 7fcc15274640 1 -- 172.21.15.46:0/2080209382 learned_addr learned my addr 172.21.15.46:0/2080209382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.285+0000 7fcc15274640 1 -- 172.21.15.46:0/2080209382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc10105e90 msgr2=0x7fcc1006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.285+0000 7fcc15274640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc10105e90 0x7fcc1006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.285+0000 7fcc15274640 1 -- 172.21.15.46:0/2080209382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc10100a60 msgr2=0x7fcc10073ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:06:12.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.285+0000 7fcc15274640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc10100a60 0x7fcc10073ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.285+0000 7fcc15274640 1 -- 172.21.15.46:0/2080209382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1006d740 con 0x7fcc10101380 2024-09-12T03:06:12.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.286+0000 7fcc15274640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c940 tx=0x7fcc0400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.287+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004220 con 0x7fcc10101380 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.287+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc040043c0 con 0x7fcc10101380 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.287+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc10071600 con 0x7fcc10101380 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.287+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc10071b50 con 0x7fcc10101380 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.287+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004a00 con 0x7fcc10101380 2024-09-12T03:06:12.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.288+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1006a840 con 0x7fcc10101380 2024-09-12T03:06:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.299+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcc040079b0 con 0x7fcc10101380 2024-09-12T03:06:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.300+0000 7fcc0effd640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 0x7fcbf80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.300+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7fcc040cf7b0 con 0x7fcc10101380 2024-09-12T03:06:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.304+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc04014030 con 0x7fcc10101380 2024-09-12T03:06:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.304+0000 7fcc15a75640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 0x7fcbf80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.310+0000 7fcc15a75640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 0x7fcbf80a53c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005c40 tx=0x7fcc00005bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:12.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:11 smithi145 ceph-mon[34674]: pgmap v503: 129 pgs: 17 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-12T03:06:12.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:11 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2098976299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:06:12.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.666+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc1006d910 con 0x7fcc10101380 2024-09-12T03:06:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.667+0000 7fcc0effd640 1 -- 172.21.15.46:0/2080209382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fcc0409bfe0 con 0x7fcc10101380 2024-09-12T03:06:12.667 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 msgr2=0x7fcbf80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 0x7fcbf80a53c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005c40 tx=0x7fcc00005bd0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 msgr2=0x7fcc1006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c940 tx=0x7fcc0400ce00 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 shutdown_connections 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fcbf80a2f00 0x7fcbf80a53c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc10105e90 0x7fcc1006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc10101380 0x7fcc1006cb60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 --2- 172.21.15.46:0/2080209382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc10100a60 0x7fcc10073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.671+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 >> 172.21.15.46:0/2080209382 conn(0x7fcc100fc6a0 msgr2=0x7fcc1010f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.672+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 shutdown_connections 2024-09-12T03:06:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.672+0000 7fcc17cd9640 1 -- 172.21.15.46:0/2080209382 wait complete. 2024-09-12T03:06:12.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T03:06:12.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.909+0000 7fc62888f640 1 -- 172.21.15.46:0/2525525208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 msgr2=0x7fc61c0b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.909+0000 7fc62888f640 1 --2- 172.21.15.46:0/2525525208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 0x7fc61c0b6fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc614009860 tx=0x7fc61402a360 comp rx=0 tx=0).stop 2024-09-12T03:06:12.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.920+0000 7fc62888f640 1 -- 172.21.15.46:0/2525525208 shutdown_connections 2024-09-12T03:06:12.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.920+0000 7fc62888f640 1 --2- 172.21.15.46:0/2525525208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0b7510 0x7fc61c0b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.920+0000 7fc62888f640 1 --2- 172.21.15.46:0/2525525208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 0x7fc61c0b6fd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.920+0000 7fc62888f640 1 --2- 172.21.15.46:0/2525525208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.920+0000 7fc62888f640 1 -- 172.21.15.46:0/2525525208 >> 172.21.15.46:0/2525525208 conn(0x7fc61c01a0d0 msgr2=0x7fc61c01a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.926+0000 7fc62888f640 1 -- 172.21.15.46:0/2525525208 shutdown_connections 2024-09-12T03:06:12.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.929+0000 7fc62888f640 1 -- 172.21.15.46:0/2525525208 wait complete. 2024-09-12T03:06:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 Processor -- start 2024-09-12T03:06:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 -- start start 2024-09-12T03:06:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 0x7fc61c0b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0b7510 0x7fc61c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0acc80 con 0x7fc61c0b7510 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0acb00 con 0x7fc61c0a5110 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.930+0000 7fc62888f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0ace00 con 0x7fc61c0a47f0 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.46:37596/0 (socket says 172.21.15.46:37596) 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 -- 172.21.15.46:0/2557351878 learned_addr learned my addr 172.21.15.46:0/2557351878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc623fff640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0b7510 0x7fc61c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 -- 172.21.15.46:0/2557351878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 msgr2=0x7fc61c0b0e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 0x7fc61c0b0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 -- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0b7510 msgr2=0x7fc61c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0b7510 0x7fc61c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.931+0000 7fc6237fe640 1 -- 172.21.15.46:0/2557351878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c0b5910 con 0x7fc61c0a47f0 2024-09-12T03:06:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.932+0000 7fc6237fe640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e9c0 tx=0x7fc61800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.937+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800ccc0 con 0x7fc61c0a47f0 2024-09-12T03:06:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.937+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6180054d0 con 0x7fc61c0a47f0 2024-09-12T03:06:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.937+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c0b5c00 con 0x7fc61c0a47f0 2024-09-12T03:06:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.938+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c158780 con 0x7fc61c0a47f0 2024-09-12T03:06:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.938+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618010430 con 0x7fc61c0a47f0 2024-09-12T03:06:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.941+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c0a5b20 con 0x7fc61c0a47f0 2024-09-12T03:06:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.943+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc618043b60 con 0x7fc61c0a47f0 2024-09-12T03:06:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.944+0000 7fc620ff9640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 0x7fc60c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.944+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7fc6180cfaf0 con 0x7fc61c0a47f0 2024-09-12T03:06:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.949+0000 7fc622ffd640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 0x7fc60c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:12.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.950+0000 7fc622ffd640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 0x7fc60c0a52a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6140095f0 tx=0x7fc614005760 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:12.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:12.953+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc61809c320 con 0x7fc61c0a47f0 2024-09-12T03:06:13.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:12 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:13.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:12 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3564542002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:12 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2080209382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:12 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:13.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:12 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3564542002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:12 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2080209382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.318+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc61c0b1ca0 con 0x7fc61c0a47f0 2024-09-12T03:06:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.319+0000 7fc620ff9640 1 -- 172.21.15.46:0/2557351878 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) ==== 103+0+11 (secure 0 0 0) 0x7fc61c0b1ca0 con 0x7fc61c0a47f0 2024-09-12T03:06:13.319 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-12T03:06:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.322+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 msgr2=0x7fc60c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.322+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 0x7fc60c0a52a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6140095f0 tx=0x7fc614005760 comp rx=0 tx=0).stop 2024-09-12T03:06:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.322+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 msgr2=0x7fc61c0b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.322+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e9c0 tx=0x7fc61800ee80 comp rx=0 tx=0).stop 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 shutdown_connections 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc60c0a2de0 0x7fc60c0a52a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc61c0b7510 0x7fc61c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc61c0a5110 0x7fc61c0b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 --2- 172.21.15.46:0/2557351878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc61c0a47f0 0x7fc61c0b0920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 >> 172.21.15.46:0/2557351878 conn(0x7fc61c01a0d0 msgr2=0x7fc61c0b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 shutdown_connections 2024-09-12T03:06:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.323+0000 7fc62888f640 1 -- 172.21.15.46:0/2557351878 wait complete. 2024-09-12T03:06:13.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:06:13.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:12 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:13.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:12 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3564542002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:12 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2080209382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.544+0000 7f8132ae4640 1 -- 172.21.15.46:0/974978418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c1095d0 msgr2=0x7f812c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.544+0000 7f8132ae4640 1 --2- 172.21.15.46:0/974978418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c1095d0 0x7f812c109a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8124009cd0 tx=0x7f812402a910 comp rx=0 tx=0).stop 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.546+0000 7f8132ae4640 1 -- 172.21.15.46:0/974978418 shutdown_connections 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.547+0000 7f8132ae4640 1 --2- 172.21.15.46:0/974978418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c11b140 0x7f812c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.547+0000 7f8132ae4640 1 --2- 172.21.15.46:0/974978418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c1095d0 0x7f812c109a10 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8124009cd0 tx=0x7f812402a910 comp rx=0 tx=0).stop 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.547+0000 7f8132ae4640 1 --2- 172.21.15.46:0/974978418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812c1091f0 0x7f812c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.547+0000 7f8132ae4640 1 -- 172.21.15.46:0/974978418 >> 172.21.15.46:0/974978418 conn(0x7f812c06cde0 msgr2=0x7f812c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.553+0000 7f8132ae4640 1 -- 172.21.15.46:0/974978418 shutdown_connections 2024-09-12T03:06:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.555+0000 7f8132ae4640 1 -- 172.21.15.46:0/974978418 wait complete. 2024-09-12T03:06:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.560+0000 7f8132ae4640 1 Processor -- start 2024-09-12T03:06:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.561+0000 7f8132ae4640 1 -- start start 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.561+0000 7f8132ae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812c1091f0 0x7f812c11aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8132ae4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c11b140 0x7f812c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8132ae4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8132ae4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f812c120c80 con 0x7f812c1091f0 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8132ae4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f812c120b00 con 0x7f812c114030 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8132ae4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f812c120e00 con 0x7f812c11b140 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8131081640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8131081640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.46:39810/0 (socket says 172.21.15.46:39810) 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.563+0000 7f8131081640 1 -- 172.21.15.46:0/2383793963 learned_addr learned my addr 172.21.15.46:0/2383793963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.564+0000 7f8131081640 1 -- 172.21.15.46:0/2383793963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c11b140 msgr2=0x7f812c113af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.564+0000 7f8131081640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c11b140 0x7f812c113af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.564+0000 7f8131081640 1 -- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812c1091f0 msgr2=0x7f812c11aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.564+0000 7f8131081640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812c1091f0 0x7f812c11aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.564+0000 7f8131081640 1 -- 172.21.15.46:0/2383793963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c1187d0 con 0x7f812c114030 2024-09-12T03:06:13.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.573+0000 7f8131081640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f812000b3f0 tx=0x7f812000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.578+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120015020 con 0x7f812c114030 2024-09-12T03:06:13.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.578+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8120011ce0 con 0x7f812c114030 2024-09-12T03:06:13.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.578+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120010dc0 con 0x7f812c114030 2024-09-12T03:06:13.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.578+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c118ac0 con 0x7f812c114030 2024-09-12T03:06:13.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.578+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c118fc0 con 0x7f812c114030 2024-09-12T03:06:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.583+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812c06dbf0 con 0x7f812c114030 2024-09-12T03:06:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.590+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8120018020 con 0x7f812c114030 2024-09-12T03:06:13.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.591+0000 7f8129ffb640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 0x7f81140a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.591+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9555+0+0 (secure 0 0 0) 0x7f81200cf2b0 con 0x7f812c114030 2024-09-12T03:06:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.591+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81200cf770 con 0x7f812c114030 2024-09-12T03:06:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.592+0000 7f8130880640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 0x7f81140a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:13.593+0000 7f8130880640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 0x7f81140a53c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f811c009710 tx=0x7f811c006c90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.046+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f812c109a10 con 0x7f812c114030 2024-09-12T03:06:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.058+0000 7f8129ffb640 1 -- 172.21.15.46:0/2383793963 <== mon.2 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740900 (secure 0 0 0) 0x7f812009bbf0 con 0x7f812c114030 2024-09-12T03:06:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T03:06:14.071 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T03:06:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 25, 2024-09-12T03:06:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 15228, 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1256678118 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1256678118 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/1256678118", 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-09-12T02:50:10.431444+0000", 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 15225, 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.080 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.082 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-12T03:06:14.089 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:14.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-12T03:06:14.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-12T03:06:14.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.101 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.104 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:06:14.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-12T03:06:14.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:06:14.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.116 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:06:14.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:06:14.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:06:14.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:06:14.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-12T03:06:14.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-12T03:06:14.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:06:14.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-12T03:06:14.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.162 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.163 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:06:14.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:06:14.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:06:14.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:06:14.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:06:14.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:06:14.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:06:14.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.177 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-12T03:06:14.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-12T03:06:14.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.186 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.187 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.204 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.207 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:06:14.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:06:14.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:06:14.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:06:14.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:06:14.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.222 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.223 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-12T03:06:14.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-12T03:06:14.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:14.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.242 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-12T03:06:14.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-12T03:06:14.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:14.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-12T03:06:14.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.258 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.260 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-12T03:06:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.266 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.273 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-12T03:06:14.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-12T03:06:14.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-12T03:06:14.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.284 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-12T03:06:14.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-12T03:06:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.295 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.296 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.301 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.302 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-12T03:06:14.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.309 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.310 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-12T03:06:14.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.316 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.317 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:06:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.328 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.336 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.339 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.342 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.344 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.346 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-12T03:06:14.351 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-12T03:06:14.352 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.353 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.360 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.362 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-12T03:06:14.364 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.365 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.368 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-12T03:06:14.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.372 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.375 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.382 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.383 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.385 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-12T03:06:14.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:14.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.413 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-12T03:06:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-12T03:06:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-12T03:06:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-12T03:06:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.445 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.455 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.456 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.463 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.468 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-12T03:06:14.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.476 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-12T03:06:14.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-12T03:06:14.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-12T03:06:14.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.492 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.493 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.501 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.508 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.512 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.513 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24908, 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-12T03:06:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.534 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-12T03:06:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.540 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-12T03:06:14.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:14.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.554 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.555 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:06:14.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.560 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-12T03:06:14.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-12T03:06:14.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-12T03:06:14.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:06:14.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:06:14.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-12T03:06:14.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-12T03:06:14.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.587 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:06:14.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:06:14.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:06:14.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-12T03:06:14.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:06:14.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.630 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.633 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-12T03:06:14.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.641 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:06:14.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:06:14.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.655 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.657 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:06:14.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.676 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.678 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.704 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:14.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.707 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.709 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.712 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-12T03:06:14.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-12T03:06:14.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-12T03:06:14.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.725 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-12T03:06:14.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-12T03:06:14.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.729 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.737 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.740 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:06:14.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-12T03:06:14.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:06:14.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-12T03:06:14.750 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.751 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.757 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.758 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.761 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.762 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:06:14.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.767 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.768 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.770 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-12T03:06:14.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.775 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.778 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.779 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-12T03:06:14.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.794 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.795 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.797 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.798 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.802 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.805 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:06:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-12T03:06:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:06:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-12T03:06:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.900 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-12T03:06:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-12T03:06:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-12T03:06:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-12T03:06:14.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-12T03:06:14.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-12T03:06:14.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.932 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.933 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.935 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.936 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.939 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.942 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.951 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.952 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.953 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.960 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:06:14.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.965 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-12T03:06:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-12T03:06:14.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.981 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.987 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.989 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.990 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:14.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.994 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:14.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:14.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-12T03:06:14.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:06:15.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.008 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:06:15.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.016 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "restful", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_schedule" 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.017 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:06:15.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.019 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.022 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-09-12T03:06:15.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-09-12T03:06:15.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.031 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-09-12T03:06:15.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:06:15.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-09-12T03:06:15.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-09-12T03:06:15.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.042 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.043 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.046 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-09-12T03:06:15.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-09-12T03:06:15.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-09-12T03:06:15.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-09-12T03:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:06:15.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.066 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:06:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-09-12T03:06:15.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-09-12T03:06:15.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-09-12T03:06:15.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-09-12T03:06:15.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-09-12T03:06:15.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.094 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:06:15.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.099 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-09-12T03:06:15.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.103 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.104 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-09-12T03:06:15.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-09-12T03:06:15.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:06:15.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:06:15.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:06:15.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.126 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.127 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.134 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:06:15.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:06:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:06:15.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:06:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.187 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.190 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:15.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-09-12T03:06:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:15.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-09-12T03:06:15.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.209 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.210 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:06:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.221 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.225 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.226 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:06:15.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.230 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.233 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.235 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:15.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-09-12T03:06:15.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.241 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.242 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.250 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.256 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.258 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.263 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.265 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-09-12T03:06:15.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-09-12T03:06:15.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-09-12T03:06:15.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.275 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.276 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.279 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.280 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.284 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-09-12T03:06:15.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.295 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-09-12T03:06:15.297 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.299 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.302 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.305 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.306 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.307 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-09-12T03:06:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-09-12T03:06:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-09-12T03:06:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-09-12T03:06:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-09-12T03:06:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-09-12T03:06:15.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.397 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.405 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.408 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.410 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.411 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.413 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.416 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-09-12T03:06:15.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-09-12T03:06:15.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-09-12T03:06:15.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-09-12T03:06:15.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.425 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.435 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.444 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.446 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.447 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.449 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.450 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-09-12T03:06:15.466 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-09-12T03:06:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-09-12T03:06:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-09-12T03:06:15.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-12T03:06:15.472 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-12T03:06:15.473 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-12T03:06:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-12T03:06:15.474 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-12T03:06:15.476 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-12T03:06:15.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "squid": [ 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-09-12T03:06:15.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 93, 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3433138603 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 8598630 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-09-12T03:06:15.480 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1534551433 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.481 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3855197802 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3362017726 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:15.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3380230591 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-12T03:06:15.483 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:13 smithi145 ceph-mon[34674]: pgmap v504: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 267 KiB/s wr, 146 op/s 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:13 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2557351878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:14 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2383793963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:13 smithi196 ceph-mon[34587]: pgmap v504: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 267 KiB/s wr, 146 op/s 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:13 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2557351878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:06:15.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:14 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2383793963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:06:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 msgr2=0x7f81140a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 0x7f81140a53c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f811c009710 tx=0x7f811c006c90 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 msgr2=0x7f812c1144e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f812000b3f0 tx=0x7f812000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 shutdown_connections 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f81140a2f00 0x7f81140a53c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f812c114030 0x7f812c1144e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f812c11b140 0x7f812c113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 --2- 172.21.15.46:0/2383793963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812c1091f0 0x7f812c11aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 >> 172.21.15.46:0/2383793963 conn(0x7f812c06cde0 msgr2=0x7f812c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 shutdown_connections 2024-09-12T03:06:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:06:14.068+0000 7f8132ae4640 1 -- 172.21.15.46:0/2383793963 wait complete. 2024-09-12T03:06:15.539 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:06:15.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:06:15.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:13 smithi046 ceph-mon[30951]: pgmap v504: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 267 KiB/s wr, 146 op/s 2024-09-12T03:06:15.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:13 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2557351878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:06:15.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:14 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2383793963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:06:15.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:15 smithi046 ceph-mon[30951]: pgmap v505: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 229 KiB/s wr, 127 op/s 2024-09-12T03:06:16.070 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:16.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:15 smithi196 ceph-mon[34587]: pgmap v505: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 229 KiB/s wr, 127 op/s 2024-09-12T03:06:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:15 smithi145 ceph-mon[34674]: pgmap v505: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 229 KiB/s wr, 127 op/s 2024-09-12T03:06:18.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:17 smithi196 ceph-mon[34587]: pgmap v506: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 229 KiB/s wr, 130 op/s 2024-09-12T03:06:18.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:17 smithi196 ceph-mon[34587]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:18.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:17 smithi046 ceph-mon[30951]: pgmap v506: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 229 KiB/s wr, 130 op/s 2024-09-12T03:06:18.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:17 smithi046 ceph-mon[30951]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:18.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:17 smithi145 ceph-mon[34674]: pgmap v506: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 229 KiB/s wr, 130 op/s 2024-09-12T03:06:18.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:17 smithi145 ceph-mon[34674]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:06:20.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:19 smithi196 ceph-mon[34587]: pgmap v507: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 188 KiB/s wr, 149 op/s 2024-09-12T03:06:20.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:19 smithi046 ceph-mon[30951]: pgmap v507: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 188 KiB/s wr, 149 op/s 2024-09-12T03:06:20.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:19 smithi145 ceph-mon[34674]: pgmap v507: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 188 KiB/s wr, 149 op/s 2024-09-12T03:06:21.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:21 smithi046 ceph-mon[30951]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:06:21.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:21 smithi196 ceph-mon[34587]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:06:21.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:21 smithi145 ceph-mon[34674]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:06:22.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:22 smithi196 ceph-mon[34587]: pgmap v509: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 129 op/s 2024-09-12T03:06:22.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:22 smithi046 ceph-mon[30951]: pgmap v509: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 129 op/s 2024-09-12T03:06:22.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:22 smithi145 ceph-mon[34674]: pgmap v509: 129 pgs: 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 115 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 129 op/s 2024-09-12T03:06:24.450 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 317, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 305, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1472160, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 187248, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.051894006, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.020706703, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.042943191, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:24.451 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 305, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.025248242, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000082781 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1472160, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 187248 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.452 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 233, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 480, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 276832, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.017937276, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012244989, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002685531, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:24.453 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 230, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.010137561, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000044076 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 276832 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-09-12T03:06:24.454 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000205495, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000124264, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-12T03:06:24.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:06:24.457 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:06:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148754, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-09-12T03:06:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-12T03:06:24.460 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 316, 2024-09-12T03:06:24.461 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1425060, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 316, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 316, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1425060, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:24.462 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T03:06:24.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:24 smithi196 ceph-mon[34587]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 130 op/s 2024-09-12T03:06:24.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:24 smithi046 ceph-mon[30951]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 130 op/s 2024-09-12T03:06:24.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:24 smithi145 ceph-mon[34674]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 130 op/s 2024-09-12T03:06:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:26 smithi196 ceph-mon[34587]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 127 op/s 2024-09-12T03:06:26.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:26 smithi196 ceph-mon[34587]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T03:06:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:26 smithi046 ceph-mon[30951]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 127 op/s 2024-09-12T03:06:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:26 smithi046 ceph-mon[30951]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T03:06:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:26 smithi145 ceph-mon[34674]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 KiB/s wr, 127 op/s 2024-09-12T03:06:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:26 smithi145 ceph-mon[34674]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T03:06:28.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:28 smithi196 ceph-mon[34587]: pgmap v513: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 90 op/s 2024-09-12T03:06:28.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:28 smithi046 ceph-mon[30951]: pgmap v513: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 90 op/s 2024-09-12T03:06:28.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:28 smithi145 ceph-mon[34674]: pgmap v513: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 90 op/s 2024-09-12T03:06:30.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:30 smithi196 ceph-mon[34587]: pgmap v514: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 149 op/s 2024-09-12T03:06:30.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:30 smithi046 ceph-mon[30951]: pgmap v514: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 149 op/s 2024-09-12T03:06:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:30 smithi145 ceph-mon[34674]: pgmap v514: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 149 op/s 2024-09-12T03:06:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:32 smithi145 ceph-mon[34674]: pgmap v515: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 131 op/s 2024-09-12T03:06:32.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:32 smithi196 ceph-mon[34587]: pgmap v515: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 131 op/s 2024-09-12T03:06:32.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:32 smithi046 ceph-mon[30951]: pgmap v515: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 131 op/s 2024-09-12T03:06:34.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:34 smithi196 ceph-mon[34587]: pgmap v516: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:34 smithi046 ceph-mon[30951]: pgmap v516: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:34.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:34 smithi145 ceph-mon[34674]: pgmap v516: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:35.103 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T03:06:35.103 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T03:06:35.103 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-12T03:06:35.103 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:06:35.104 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:06:35.104 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-12T03:06:35.104 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:06:35.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:06:35.401 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:36 smithi196 ceph-mon[34587]: pgmap v517: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:36.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:36 smithi196 ceph-mon[34587]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T03:06:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:36 smithi046 ceph-mon[30951]: pgmap v517: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:36 smithi046 ceph-mon[30951]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T03:06:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:36 smithi145 ceph-mon[34674]: pgmap v517: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 137 op/s 2024-09-12T03:06:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:36 smithi145 ceph-mon[34674]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T03:06:38.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:38 smithi196 ceph-mon[34587]: pgmap v519: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 135 op/s 2024-09-12T03:06:38.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:38 smithi046 ceph-mon[30951]: pgmap v519: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 135 op/s 2024-09-12T03:06:38.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:38 smithi145 ceph-mon[34674]: pgmap v519: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 135 op/s 2024-09-12T03:06:40.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:40 smithi196 ceph-mon[34587]: pgmap v520: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 79 op/s 2024-09-12T03:06:40.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:40 smithi046 ceph-mon[30951]: pgmap v520: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 79 op/s 2024-09-12T03:06:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:40 smithi145 ceph-mon[34674]: pgmap v520: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 79 op/s 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 322, 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 312, 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1471664, 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 192560, 2024-09-12T03:06:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.052034448, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.020199397, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.043159618, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 311, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.019260502, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000061930 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1471664, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 192560 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 7, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 242, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3424, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 278512, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-09-12T03:06:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.019771498, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.011902350, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.004863413, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 240, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.020945697, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000087273 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 278512 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:06:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000312548, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000235587, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-12T03:06:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-12T03:06:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:06:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 147930, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 28, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-12T03:06:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-12T03:06:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 325, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1426230, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 325, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-12T03:06:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 325, 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1426230, 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:41.348 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T03:06:42.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:42 smithi196 ceph-mon[34587]: pgmap v521: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-12T03:06:42.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:42 smithi046 ceph-mon[30951]: pgmap v521: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-12T03:06:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:42 smithi145 ceph-mon[34674]: pgmap v521: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 73 op/s 2024-09-12T03:06:44.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:44 smithi196 ceph-mon[34587]: pgmap v522: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:44.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:44 smithi046 ceph-mon[30951]: pgmap v522: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:44.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:44 smithi145 ceph-mon[34674]: pgmap v522: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:46.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:46 smithi196 ceph-mon[34587]: pgmap v523: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:46.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:46 smithi046 ceph-mon[30951]: pgmap v523: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:46.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:46 smithi145 ceph-mon[34674]: pgmap v523: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:06:48.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:48 smithi196 ceph-mon[34587]: pgmap v524: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T03:06:48.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:48 smithi046 ceph-mon[30951]: pgmap v524: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T03:06:48.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:48 smithi145 ceph-mon[34674]: pgmap v524: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-12T03:06:50.128 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:06:50.128 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:06:50.128 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T03:06:50.128 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:06:50.128 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:06:50.129 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T03:06:50.129 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:06:50.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:06:50.394 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:50.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:50 smithi196 ceph-mon[34587]: pgmap v525: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:06:50.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:50 smithi046 ceph-mon[30951]: pgmap v525: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:06:50.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:50 smithi145 ceph-mon[34674]: pgmap v525: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:06:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:52 smithi196 ceph-mon[34587]: pgmap v526: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:52.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:52 smithi046 ceph-mon[30951]: pgmap v526: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:52 smithi145 ceph-mon[34674]: pgmap v526: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:54.699 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:54 smithi046 ceph-mon[30951]: pgmap v527: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:06:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:54 smithi196 ceph-mon[34587]: pgmap v527: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:06:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:54 smithi145 ceph-mon[34674]: pgmap v527: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:06:56.037 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2944, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.003476360, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000048125, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002389882, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:56.038 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.004250661, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001416887 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:06:56.039 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 322, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 312, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1471664, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 192560, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.054190086, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.020389928, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.046400425, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 311, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012966405, 2024-09-12T03:06:56.040 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000041692 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1471664, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 192560 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 239, 2024-09-12T03:06:56.041 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 480, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 277888, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.017186434, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.011867610, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002405667, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 236, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.022679447, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000096099 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.042 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 277888 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:06:56.043 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:06:56.044 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:06:56.045 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148748, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 32, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-12T03:06:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-12T03:06:56.047 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 325, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1426230, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 325, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-12T03:06:56.048 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 325, 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1426230, 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:06:56.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T03:06:56.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:56 smithi196 ceph-mon[34587]: pgmap v528: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:56 smithi046 ceph-mon[30951]: pgmap v528: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:56.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:56 smithi145 ceph-mon[34674]: pgmap v528: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:58.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:58 smithi196 ceph-mon[34587]: pgmap v529: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:06:58.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:58 smithi046 ceph-mon[30951]: pgmap v529: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:06:58.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:58 smithi145 ceph-mon[34674]: pgmap v529: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-12T03:06:59.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:06:59 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:59.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:06:59 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:59.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:06:59 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:00.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:00 smithi196 ceph-mon[34587]: pgmap v530: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:07:00.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:00 smithi046 ceph-mon[30951]: pgmap v530: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:07:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:00 smithi145 ceph-mon[34674]: pgmap v530: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:07:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:01 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:01 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:01.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:01 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:07:01.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:01 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:01.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:01 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:01.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:01 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:07:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:01 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:01 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:01.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:01 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:07:02.020 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-12T03:07:02.020 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:07:02.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:07:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:02.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:02 smithi196 ceph-mon[34587]: pgmap v531: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:02.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:02 smithi145 ceph-mon[34674]: pgmap v531: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:02.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:02 smithi046 ceph-mon[30951]: pgmap v531: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:03.367 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-12T03:07:03.367 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001292535, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000184037, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:07:03.368 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 333, 2024-09-12T03:07:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 321, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1476512, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 197488, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.059234361, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.021616782, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.049977982, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 321, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.055197322, 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000171954 2024-09-12T03:07:03.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1476512, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 197488 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 246, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 480, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 279120, 2024-09-12T03:07:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.017839260, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012163880, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002753717, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 243, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.014709304, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000060532 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:07:03.372 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 279120 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:03.373 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:03.374 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:03.375 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 147970, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 29, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:03.376 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:03.377 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 332, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1427140, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 332, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 332, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1427140, 2024-09-12T03:07:03.378 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-12T03:07:03.379 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:07:04.523 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T03:07:04.523 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:07:04.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:07:04.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:04 smithi196 ceph-mon[34587]: pgmap v532: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:07:04.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:04 smithi046 ceph-mon[30951]: pgmap v532: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:07:04.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:04 smithi145 ceph-mon[34674]: pgmap v532: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:07:04.866 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:07:05.585 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-12T03:07:05.585 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2449, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2625, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2133187, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 16858199, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 17, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.254964503, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.091426028, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.095307530, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.065183421, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2621, 2024-09-12T03:07:05.586 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.155482422, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000059321 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 25, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000002918, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000116 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1630240, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 660416 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 5969, 2024-09-12T03:07:05.587 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 9334, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2549388, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 4418877, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 12, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.578077849, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.213252512, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.380225838, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.020261509, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9333, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.613450299, 2024-09-12T03:07:05.588 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000065729 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 13141, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.002864695, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000217 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 128080 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:05.589 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2752, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 3111, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1284870, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 5519767, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.247441416, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.073349683, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.102756690, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.065687955, 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:05.590 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3111, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.246447234, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000079218 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2378, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000473773, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000199 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-12T03:07:05.591 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3441, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.189576090, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000345706 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-12T03:07:05.592 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2023, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.100050681, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000049456 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 1713, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 1505, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1505, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 203.921878864, 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.135496265 2024-09-12T03:07:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 208, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 2, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 35, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 3, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 1538, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 343, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 1098, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 97, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 1047, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 413, 2024-09-12T03:07:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 4, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 4, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 4, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 2594, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 2269, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 183, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 7, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 1, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 13, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 366, 2024-09-12T03:07:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 23, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 1210, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 201, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 353386546, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 1, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 17, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 404, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 1580, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 1923, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 1434, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 10, 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 754 2024-09-12T03:07:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-12T03:07:05.598 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-12T03:07:05.598 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 3, 2024-09-12T03:07:05.598 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 2, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 13, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 16, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 83, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 319, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 236, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-12T03:07:05.599 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 14, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 3, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-12T03:07:05.600 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 2901, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 2901, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 7, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 7, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 24106708, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1511, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 19.148285235, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.012672591 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 994, 2024-09-12T03:07:05.602 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 1538, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 544, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 129, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 210, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 81, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 1538, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 2200, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 662, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 4, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 585, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 581, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 64816, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 223520 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.603 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 2181, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 155, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 1713, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 33, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 326, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 111, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.140407712, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001264934 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 414, 2024-09-12T03:07:05.604 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 20.298173425, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.049029404 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-12T03:07:05.605 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 384, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 103.537469886, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.269628827 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.608 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-12T03:07:05.608 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.608 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.608 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-12T03:07:05.609 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 239, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 74.910657877, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.313433714 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 38, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.039599064, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001042080 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.610 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.611 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 83, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4.552333773, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.054847394 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.612 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.613 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.614 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 236, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.443237127, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001878123 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:05.615 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 3, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 1, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 883, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 883, 2024-09-12T03:07:05.616 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 1012, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:05.617 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:05.618 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:05.619 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 955129, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 272, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9392, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 189, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 3548335, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 35155, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-12T03:07:05.621 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 4094, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 20202818, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 32, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 4094, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4094, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 261.680585180, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.063918071 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4094, 2024-09-12T03:07:05.622 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4580 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 4094, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 13, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 4081, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 10, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 9, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 1520, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 69, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-12T03:07:05.623 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 1, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 2042, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 20, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:05.624 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-12T03:07:05.625 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 116, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 28, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2024-09-12T03:07:05.626 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 15, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 3, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 697, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 6, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 206 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-12T03:07:05.627 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 12 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 66, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 8, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 2024, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 236, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:05.628 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 11166, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 4938255, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 11166, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 11166, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4938255, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.629 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 4094, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 20241603, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 4094, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 20241603, 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:05.630 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 4094, 2024-09-12T03:07:05.631 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 4094, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 4094, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4094, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 236, 2024-09-12T03:07:05.632 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 38876, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 236, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 9, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 38876, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:05.633 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x558eb3aec960": { 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2901, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 19912404, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2901, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 1511, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 19912404, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:05.634 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:05.635 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.635 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:05.635 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds.exported=13 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds.imported=23 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=1 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:07:06.665 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2024-09-12T03:07:06.666 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:07:06.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:07:06.690 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:06 smithi145 ceph-mon[34674]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:07:06.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:06 smithi196 ceph-mon[34587]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:07:06.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:06 smithi046 ceph-mon[30951]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:07:06.920 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:07:07.562 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-12T03:07:07.562 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:07.562 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 329, 2024-09-12T03:07:07.562 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 319, 2024-09-12T03:07:07.562 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1473568, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 197040, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.051030150, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.020840445, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.042367272, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 318, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.017753481, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000055828 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.563 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1473568, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 197040 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 245, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 480, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 278944, 2024-09-12T03:07:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.015196737, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.010429839, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.002083916, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 242, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.021795339, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000090063 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 278944 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 4, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 4, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2944, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 448, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.002234192, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000028657, 2024-09-12T03:07:07.566 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.002194188, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.004009904, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001336634 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.567 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:07.568 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:07.569 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:07.570 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 147960, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 28, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2024-09-12T03:07:07.571 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:07.572 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 332, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1427140, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 332, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 332, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1427140, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:07.573 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:07.574 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-12T03:07:08.345 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-12T03:07:08.345 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:07:08.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:07:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:08 smithi145 ceph-mon[34674]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-12T03:07:08.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:08 smithi145 ceph-mon[34674]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T03:07:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:08 smithi196 ceph-mon[34587]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-12T03:07:08.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:08 smithi196 ceph-mon[34587]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T03:07:08.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:08 smithi046 ceph-mon[30951]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-12T03:07:08.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:08 smithi046 ceph-mon[30951]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T03:07:08.937 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:07:09.675 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-12T03:07:09.675 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:09.675 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3571, 2024-09-12T03:07:09.675 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 3573, 2024-09-12T03:07:09.675 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 16724600, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 1135585, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.282864112, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.081893490, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.087345982, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.093148898, 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:09.676 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3571, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.121875254, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000034129 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.677 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 698, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 663, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 5891968, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 300974, 2024-09-12T03:07:09.678 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.081193430, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.027099087, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.057795622, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.009322699, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 662, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.029665302, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000044811 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.679 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1618464, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 201696 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 62, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 301, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 16976, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 688146, 2024-09-12T03:07:09.680 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 9, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.020831143, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.010779442, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.005592658, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.001759846, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 299, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.024713172, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000082652 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:09.681 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 670656 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-12T03:07:09.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6762, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.146142497, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000021612 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-09-12T03:07:09.683 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.684 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 1780, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 239, 2024-09-12T03:07:09.685 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 1541, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 1439, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 3, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-09-12T03:07:09.686 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:09.687 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:09.688 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-09-12T03:07:09.690 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:09.691 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 2901, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 7, 2024-09-12T03:07:09.692 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 24106708, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 24106134, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 2901 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 986, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 1518, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 532, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 119, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 201, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 82, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 1780, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 2179, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 399, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-09-12T03:07:09.694 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 57024, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215328 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.695 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.696 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.697 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-09-12T03:07:09.697 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.697 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.698 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-09-12T03:07:09.699 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.701 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.702 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-09-12T03:07:09.703 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.704 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.704 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.704 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-09-12T03:07:09.704 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.705 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.705 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.705 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-09-12T03:07:09.705 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-09-12T03:07:09.706 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.708 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:09.709 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.711 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-09-12T03:07:09.712 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 3, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3267, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:09.713 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:09.714 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:09.715 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 199071, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 59, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:09.716 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9752, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 198, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 3636346, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 29823, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:09.717 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 3938, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 3938, 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-09-12T03:07:09.718 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3938, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 7.350840953, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001866643 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3938, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3939 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 3938, 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 3938, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 2170, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 1766, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-09-12T03:07:09.720 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-09-12T03:07:09.721 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-09-12T03:07:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 117, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 29, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 6, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 7, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 1, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-09-12T03:07:09.723 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.724 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:09.725 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 4327, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 22221517, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 4327, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 4327, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 22221517, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.726 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 3938, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 19922606, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3938, 2024-09-12T03:07:09.727 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 19922606, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:09.728 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 3938, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 3938, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 3938, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 3938, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.729 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:09.730 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.731 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.732 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55896e4a8b40": { 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-09-12T03:07:09.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-09-12T03:07:09.734 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:09.735 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-12T03:07:10.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:10 smithi196 ceph-mon[34587]: pgmap v536: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:10 smithi046 ceph-mon[30951]: pgmap v536: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:10.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:10 smithi145 ceph-mon[34674]: pgmap v536: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:12.445 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:07:12.445 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:07:12.445 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-12T03:07:12.445 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:07:12.446 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:07:12.446 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T03:07:12.446 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:07:12.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:07:12.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:12 smithi196 ceph-mon[34587]: pgmap v537: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:12.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:12 smithi046 ceph-mon[30951]: pgmap v537: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:12.852 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:07:12.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:12 smithi145 ceph-mon[34674]: pgmap v537: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 332, 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 322, 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1474384, 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 198960, 2024-09-12T03:07:13.489 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.055035400, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.020921134, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.047247677, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 321, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.021725340, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000067680 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:13.490 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1474384, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 198960 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 248, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 480, 2024-09-12T03:07:13.491 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 279472, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.015190485, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.010502764, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.001806314, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 245, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.023419614, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000095590 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.492 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 279472 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 4, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 4, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2944, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 448, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2024-09-12T03:07:13.493 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.002208340, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000029930, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.002148853, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.003947018, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001315672 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:13.495 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:13.496 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:13.497 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 147960, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 28, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:13.498 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-09-12T03:07:13.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 335, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1427530, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 335, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 335, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1427530, 2024-09-12T03:07:13.500 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-12T03:07:13.501 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:07:14.231 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-12T03:07:14.231 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:07:14.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:07:14.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:14 smithi046 ceph-mon[30951]: pgmap v538: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:14.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:14 smithi145 ceph-mon[34674]: pgmap v538: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:14.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:14 smithi196 ceph-mon[34587]: pgmap v538: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:15.414 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:07:16.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:16 smithi196 ceph-mon[34587]: pgmap v539: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:16.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:16 smithi046 ceph-mon[30951]: pgmap v539: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:16.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:16 smithi145 ceph-mon[34674]: pgmap v539: 129 pgs: 18 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 56 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:18.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:18 smithi046 ceph-mon[30951]: pgmap v540: 129 pgs: 13 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:18.862 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:18 smithi196 ceph-mon[34587]: pgmap v540: 129 pgs: 13 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:18.862 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:18 smithi145 ceph-mon[34674]: pgmap v540: 129 pgs: 13 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 75 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2627, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2629, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2350141, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 3080585, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.290527523, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.100568511, 2024-09-12T03:07:18.909 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.134580430, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.063107669, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2631, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.172958337, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000065738 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1899, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000400905, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000211 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1629632, 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 210432 2024-09-12T03:07:18.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2075, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1958, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1083487, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1647326, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 12, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.161696896, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.055400166, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.089348093, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.025012131, 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:18.911 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1955, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.126963635, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000064943 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2422, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000523199, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000216 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 585552 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1824, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2547, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 596124, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1309725, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 15, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.186746750, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.056757124, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.119277967, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.013725688, 2024-09-12T03:07:18.913 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2543, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.179622682, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000070634 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2957, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000775544, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000262 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3776, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 18912 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 958, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.272958689, 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000284925 2024-09-12T03:07:18.915 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1632, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.068287196, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000041842 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 643, 2024-09-12T03:07:18.916 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 622, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 622, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 208.983855107, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.335986905 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 21, 2024-09-12T03:07:18.917 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 9, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 19, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 1, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 11, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 10, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 587, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-09-12T03:07:18.918 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 1041, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 898, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 19, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 9, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 13, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 506, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 6, 2024-09-12T03:07:18.919 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 273, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 201, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 353386546, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 1, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:18.920 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 9, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 264, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 305, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 811, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 470, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 3, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 89 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-12T03:07:18.921 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 3, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 2, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 14, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 4, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 42, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 42, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:18.922 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 13, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 1, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 1238, 2024-09-12T03:07:18.923 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-12T03:07:18.924 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-12T03:07:18.924 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1238, 2024-09-12T03:07:18.924 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 3, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-12T03:07:18.925 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 5996354, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 349, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 11.915633371, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.034142215 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.926 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 15, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 533, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 518, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 14, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 97, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 83, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 11, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 640, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 629, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-09-12T03:07:18.927 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 197, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 196, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 56952, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 832, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 14, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 643, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 29, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 91, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.017835134, 2024-09-12T03:07:18.928 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003567026 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.670761379, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.111793563 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-12T03:07:18.929 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 285, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 133.564569278, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.468647611 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.930 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.933 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-12T03:07:18.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 266, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 74.623108162, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.280538000 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 18, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021177282, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001176515 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.936 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-12T03:07:18.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-12T03:07:18.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 42, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.086403872, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002057235 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 146, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 73, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1001, 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:18.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:18.943 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 282524, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 112, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9752, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 198, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:18.946 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 65632, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 269, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-12T03:07:18.947 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2239, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 1910883, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2239, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2239, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 244.535821767, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.109216534 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2239, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2500 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2239, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 18, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 2221, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 350, 2024-09-12T03:07:18.949 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 40, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 1625, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-12T03:07:18.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-12T03:07:18.951 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-12T03:07:18.952 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 117, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 29, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 15, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 3, 2024-09-12T03:07:18.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 325, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 26, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 103 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 9 2024-09-12T03:07:18.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 52, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 1, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 1612, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 42, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.955 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 6522, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 3417890, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 6522, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:18.956 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 6522, 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3417890, 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:18.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2239, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1917239, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2239, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1917239, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:18.960 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2239, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2239, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2239, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2239, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:18.961 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 42, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 6730, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 42, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 6730, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.962 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x564e174cc960": { 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1238, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1802050, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1238, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 349, 2024-09-12T03:07:18.963 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1802050, 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:18.964 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-12T03:07:20.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:20 smithi046 ceph-mon[30951]: pgmap v541: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 180 B/s wr, 6 op/s 2024-09-12T03:07:20.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:20 smithi046 ceph-mon[30951]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T03:07:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:20 smithi145 ceph-mon[34674]: pgmap v541: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 180 B/s wr, 6 op/s 2024-09-12T03:07:20.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:20 smithi145 ceph-mon[34674]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T03:07:20.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:20 smithi196 ceph-mon[34587]: pgmap v541: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 180 B/s wr, 6 op/s 2024-09-12T03:07:20.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:20 smithi196 ceph-mon[34587]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T03:07:22.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:22 smithi046 ceph-mon[30951]: pgmap v543: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:07:22.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:22 smithi145 ceph-mon[34674]: pgmap v543: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:07:22.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:22 smithi196 ceph-mon[34587]: pgmap v543: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:07:23.655 INFO:tasks.check_counter:Daemon mds.c mds.exported=13 2024-09-12T03:07:23.655 INFO:tasks.check_counter:Daemon mds.c mds.imported=6 2024-09-12T03:07:23.655 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=1 2024-09-12T03:07:23.656 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:07:23.656 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:07:23.656 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=1 2024-09-12T03:07:23.656 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:07:23.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:07:23.919 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1418, 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1420, 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4640847, 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 450934, 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:24.554 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.145144537, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.043560998, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.040367189, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.047335317, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1418, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.069034855, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000048684 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.555 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1622, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1864, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 5398550, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1101792, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-12T03:07:24.556 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.145058468, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.048650331, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.038453406, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.046071808, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1861, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.093221057, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000050091 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 587520 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 858, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 821, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 5934805, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 352279, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.097519173, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.037480866, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.061219699, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.010689198, 2024-09-12T03:07:24.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 821, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.038748035, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000047196 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1623440, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 205680 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.559 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5888, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.165823479, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000028162 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:24.560 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.561 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 579, 2024-09-12T03:07:24.562 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 371, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 208, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 49, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:07:24.563 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-09-12T03:07:24.564 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 7, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:24.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:24.566 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-12T03:07:24.567 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-12T03:07:24.568 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 4823, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 72, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 17180940, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 17570170, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 17570034, 2024-09-12T03:07:24.569 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 4823 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 9, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1149, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1140, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 8, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 136, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 128, 2024-09-12T03:07:24.570 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 579, 2024-09-12T03:07:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 1448, 2024-09-12T03:07:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 869, 2024-09-12T03:07:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-12T03:07:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-12T03:07:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 51524, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-12T03:07:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:24.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.583 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.583 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-12T03:07:24.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-12T03:07:24.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 6, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 4, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 4, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 3269, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.586 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:24.587 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:24.588 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 189433, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 52, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:24.589 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 192, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:24.590 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 455680, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 2438, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3496, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 26, 2024-09-12T03:07:24.591 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 4, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3496, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3496, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 97.822846244, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.027981363 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-12T03:07:24.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3496, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3497 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3496, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3496, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 2094, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1399, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-12T03:07:24.593 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-12T03:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-12T03:07:24.595 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 118, 2024-09-12T03:07:24.596 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 30, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 5, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-12T03:07:24.597 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.598 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3894, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 15600836, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3894, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3894, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 15600836, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.599 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3496, 2024-09-12T03:07:24.600 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 13375876, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3496, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13375876, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:24.601 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:24.602 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:24.602 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3496, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3496, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3496, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3496, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:24.604 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x5597fbb98b40": { 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:24.605 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:24.606 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-12T03:07:24.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:24 smithi046 ceph-mon[30951]: pgmap v544: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:24.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:24 smithi145 ceph-mon[34674]: pgmap v544: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:24.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:24 smithi196 ceph-mon[34587]: pgmap v544: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:07:25.708 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T03:07:25.708 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:07:25.709 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:07:25.965 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 3371, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3247, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1244762, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 11652854, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 145, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.370788260, 2024-09-12T03:07:26.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.118997860, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.155912591, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.084832520, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3246, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.412541620, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000127092 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2120, 2024-09-12T03:07:26.496 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000428105, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000201 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 46080, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 3344 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 10015, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 11711, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3399921, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 12448503, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 17, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-12T03:07:26.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.783074089, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.260508890, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.411792661, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.138151244, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 11708, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.730815579, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000062420 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16396, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.003018556, 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000184 2024-09-12T03:07:26.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 93024, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 635296 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 6480, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4873, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4097551, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 4139524, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 17, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-12T03:07:26.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.572200393, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.166328797, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.300302401, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.137687544, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4872, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.164723895, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000239064 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5825, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001316923, 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000226 2024-09-12T03:07:26.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2170160, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 383840 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3251, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.027258310, 2024-09-12T03:07:26.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000623579 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2603, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.151572448, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000058229 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 3745, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 3377, 2024-09-12T03:07:26.502 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3377, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 259.801696294, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.076932690 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 368, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 2, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 568, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 12, 2024-09-12T03:07:26.503 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 11, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 11, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 829, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 2, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 4, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 5534, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 4789, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 368, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 8, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:07:26.504 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 53, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 13, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 771, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 10, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 160, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-09-12T03:07:26.505 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 7, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 1275, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 1563, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 6, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:26.506 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 3326, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 1218, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 17, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 2042 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 2, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 12, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 19, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-12T03:07:26.507 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 573, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 512, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 61, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:26.508 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:26.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:26.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 13, 2024-09-12T03:07:26.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-12T03:07:26.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:07:26.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 4823, 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 4642, 2024-09-12T03:07:26.510 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 4642, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 181, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 1, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 77, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 76, 2024-09-12T03:07:26.511 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 76, 2024-09-12T03:07:26.512 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 1, 2024-09-12T03:07:26.512 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 1, 2024-09-12T03:07:26.512 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-12T03:07:26.512 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 1, 2024-09-12T03:07:26.514 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 17180940, 2024-09-12T03:07:26.514 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 17570170, 2024-09-12T03:07:26.514 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-12T03:07:26.514 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-12T03:07:26.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1056, 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 55.151706454, 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.052226994 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.515 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-12T03:07:26.516 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 16, 2024-09-12T03:07:26.516 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1170, 2024-09-12T03:07:26.516 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1154, 2024-09-12T03:07:26.516 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 15, 2024-09-12T03:07:26.516 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 167, 2024-09-12T03:07:26.517 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 152, 2024-09-12T03:07:26.517 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 12, 2024-09-12T03:07:26.517 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 1489, 2024-09-12T03:07:26.517 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 1477, 2024-09-12T03:07:26.518 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 2, 2024-09-12T03:07:26.518 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 878, 2024-09-12T03:07:26.518 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 876, 2024-09-12T03:07:26.518 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 64280, 2024-09-12T03:07:26.518 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 231712 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 4551, 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 1, 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 3745, 2024-09-12T03:07:26.519 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 52, 2024-09-12T03:07:26.520 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 90, 2024-09-12T03:07:26.520 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-12T03:07:26.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 649, 2024-09-12T03:07:26.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.642390633, 2024-09-12T03:07:26.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000989816 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 899, 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.212280093, 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000236129 2024-09-12T03:07:26.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.522 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:26.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 61, 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.117965200, 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001933855 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.523 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-12T03:07:26.524 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 444, 2024-09-12T03:07:26.524 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 190.612711475, 2024-09-12T03:07:26.524 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.429307908 2024-09-12T03:07:26.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.524 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:26.525 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.525 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.525 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.525 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-12T03:07:26.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.527 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.529 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.529 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-12T03:07:26.529 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 17, 2024-09-12T03:07:26.530 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002463863, 2024-09-12T03:07:26.530 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000144933 2024-09-12T03:07:26.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.530 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-12T03:07:26.530 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 105, 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.098342599, 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000936596 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.531 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 17, 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 12.897044314, 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.758649665 2024-09-12T03:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 328, 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 37.849967230, 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.115396241 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.533 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 164, 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.529293238, 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003227397 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 113, 2024-09-12T03:07:26.534 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.968853123, 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.043972151 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.099988945, 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.006249309 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:26.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-09-12T03:07:26.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 10.972749591, 2024-09-12T03:07:26.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 1.097274959 2024-09-12T03:07:26.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:26.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.005956532, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001191306 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001103845, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000275961 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021095525, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002636940 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 13, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010029565, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000771505 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 524, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.759460523, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001449352 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.542 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 6, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 4, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 100, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 50, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1007, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.543 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:26.544 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:26.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 306548, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 122, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 176, 2024-09-12T03:07:26.546 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 2, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 192, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 81176, 2024-09-12T03:07:26.547 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 366, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 6361, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 15784834, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 34, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 6361, 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-12T03:07:26.548 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6361, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 386.604700785, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.060777346 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6361, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 9005 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 6361, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 45, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 6316, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-12T03:07:26.549 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 467, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 820, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 28, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 1084, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 266, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 3247, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 1288, 2024-09-12T03:07:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 6, 2024-09-12T03:07:26.551 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-12T03:07:26.552 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 118, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 36, 2024-09-12T03:07:26.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 1589, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 19, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 191 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 4 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 52, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 10, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 2729, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 512, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:26.555 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 19862, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 6859893, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 19862, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 19862, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 6859893, 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:26.556 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:26.558 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:26 smithi196 ceph-mon[34587]: pgmap v545: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 6361, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 15842126, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 6361, 2024-09-12T03:07:26.558 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 15842126, 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.559 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 6361, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 6361, 2024-09-12T03:07:26.560 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 6361, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 6361, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 512, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 57760, 2024-09-12T03:07:26.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 512, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 28, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 57760, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x555abee4c960": { 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:26.562 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 4823, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 13375866, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 4823, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1056, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13375866, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:26.563 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:26.564 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-12T03:07:26.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:26 smithi046 ceph-mon[30951]: pgmap v545: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:26.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:26 smithi145 ceph-mon[34674]: pgmap v545: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds.exported=13 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=17 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=10 2024-09-12T03:07:27.340 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-12T03:07:27.340 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:07:27.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:07:27.590 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 398, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 363, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1622530, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 208596, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 1, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.059420456, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.022973643, 2024-09-12T03:07:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.052072815, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000017126, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 362, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.020162314, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000055697 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1621696, 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 208272 2024-09-12T03:07:28.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1039, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1286, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 257404, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 918787, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.110420691, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.034207241, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.026291517, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.039283462, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1284, 2024-09-12T03:07:28.115 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.064000072, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000049844 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 590288 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.116 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2475, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2477, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2531839, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 787108, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.232133358, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.068215438, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.064303105, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.079682380, 2024-09-12T03:07:28.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2475, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.114822184, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000046392 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.118 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5926, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.159747318, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000026957 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-12T03:07:28.119 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 43, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 43, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 43, 2024-09-12T03:07:28.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-12T03:07:28.122 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:07:28.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-12T03:07:28.124 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:07:28.125 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-12T03:07:28.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1238, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-12T03:07:28.127 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 5996354, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 5990173, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 1238 2024-09-12T03:07:28.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 4, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 479, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 475, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 2, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 87, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 85, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 43, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 579, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 536, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-12T03:07:28.129 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50632, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.132 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-12T03:07:28.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.134 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-12T03:07:28.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.135 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.135 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-12T03:07:28.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-12T03:07:28.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 2151, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.143 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:07:28.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-12T03:07:28.145 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 190939, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 55, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-12T03:07:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9512, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 192, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 45488, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 206, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-12T03:07:28.147 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3509, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3509, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3509, 2024-09-12T03:07:28.148 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 8.619810202, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002456486 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3509, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3510 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3509, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3509, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 2171, 2024-09-12T03:07:28.149 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1336, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:07:28.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-12T03:07:28.151 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 118, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-12T03:07:28.152 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 30, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 6, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-12T03:07:28.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:28.154 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3908, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 4036933, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3908, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3908, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 4036933, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.155 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3509, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1802055, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3509, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1802055, 2024-09-12T03:07:28.156 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:28.157 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.157 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-12T03:07:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3509, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3509, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3509, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3509, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.161 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:28.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55a234c56b40": { 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-12T03:07:28.163 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-12T03:07:28.164 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-12T03:07:28.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:28 smithi046 ceph-mon[30951]: pgmap v546: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:28.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:28 smithi145 ceph-mon[34674]: pgmap v546: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:28.928 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:28 smithi196 ceph-mon[34587]: pgmap v546: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:28.955 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T03:07:28.955 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T03:07:28.956 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-12T03:07:28.956 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:07:28.956 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:07:28.956 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-12T03:07:28.956 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-12T03:07:28.966 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-12T03:07:28.967 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T03:07:28.967 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:07:29.008 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T03:07:29.009 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:07:29.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:07:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/3013289251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 msgr2=0x7fdcbc112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/3013289251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc112650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac02a480 comp rx=0 tx=0).stop 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/3013289251 shutdown_connections 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/3013289251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc112b90 0x7fdcbc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/3013289251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc112650 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/3013289251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc074590 0x7fdcbc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/3013289251 >> 172.21.15.46:0/3013289251 conn(0x7fdcbc0fd8c0 msgr2=0x7fdcbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/3013289251 shutdown_connections 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.897+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/3013289251 wait complete. 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 Processor -- start 2024-09-12T03:07:30.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 -- start start 2024-09-12T03:07:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc074590 0x7fdcbc1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc112b90 0x7fdcbc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc1183f0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc118270 con 0x7fdcbc074590 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc3cb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc118570 con 0x7fdcbc112b90 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47980/0 (socket says 172.21.15.46:47980) 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 -- 172.21.15.46:0/2221798437 learned_addr learned my addr 172.21.15.46:0/2221798437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc2256640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc112b90 0x7fdcbc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1a55640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc074590 0x7fdcbc1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 -- 172.21.15.46:0/2221798437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc112b90 msgr2=0x7fdcbc19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc112b90 0x7fdcbc19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 -- 172.21.15.46:0/2221798437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc074590 msgr2=0x7fdcbc1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:30.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc074590 0x7fdcbc1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.898+0000 7fdcc1254640 1 -- 172.21.15.46:0/2221798437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc19efd0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcc1254640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdcac005940 tx=0x7fdcac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac03f030 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc19f260 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcac003cb0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac035070 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.899+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1ab440 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.900+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc075cd0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.901+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdcac042030 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.902+0000 7fdcb2ffd640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.902+0000 7fdcc1a55640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.902+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7fdcac0f01e0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.902+0000 7fdcc1a55640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 0x7fdca80a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009bd0 tx=0x7fdcb8005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:29.905+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcac0bc3e0 con 0x7fdcbc074eb0 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/328833784 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3748867549 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/829485206 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2351168704 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2520496515 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1895867717 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1022575768 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/791127312 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2423476506 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3748867549 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2936988098 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2008665137 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:30.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.093+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdcbc074970 con 0x7fdcbc074eb0 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.094+0000 7fdcb2ffd640 1 -- 172.21.15.46:0/2221798437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v118) ==== 81+0+943 (secure 0 0 0) 0x7fdcbc074970 con 0x7fdcbc074eb0 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:listed 16 entries 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 msgr2=0x7fdca80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 0x7fdca80a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009bd0 tx=0x7fdcb8005f90 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 msgr2=0x7fdcbc1a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdcac005940 tx=0x7fdcac0042f0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 shutdown_connections 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcbc112b90 0x7fdcbc19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc074eb0 0x7fdcbc1a4bd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 --2- 172.21.15.46:0/2221798437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcbc074590 0x7fdcbc1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 >> 172.21.15.46:0/2221798437 conn(0x7fdcbc0fd8c0 msgr2=0x7fdcbc0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 shutdown_connections 2024-09-12T03:07:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:30.096+0000 7fdcc3cb9640 1 -- 172.21.15.46:0/2221798437 wait complete. 2024-09-12T03:07:30.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:30 smithi046 ceph-mon[30951]: pgmap v547: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:07:30.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:30 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2221798437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:30 smithi145 ceph-mon[34674]: pgmap v547: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:07:30.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:30 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2221798437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:30.915 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-12T03:07:30.915 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:07:30.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:07:30.968 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:30 smithi196 ceph-mon[34587]: pgmap v547: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:07:30.968 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:30 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2221798437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 -- 172.21.15.46:0/2013170267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 msgr2=0x7f1144100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 --2- 172.21.15.46:0/2013170267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f1144100db0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1134009900 tx=0x7f113402a410 comp rx=0 tx=0).stop 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 -- 172.21.15.46:0/2013170267 shutdown_connections 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 --2- 172.21.15.46:0/2013170267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 0x7f1144110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 --2- 172.21.15.46:0/2013170267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 --2- 172.21.15.46:0/2013170267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f1144100db0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 -- 172.21.15.46:0/2013170267 >> 172.21.15.46:0/2013170267 conn(0x7f11440fc7d0 msgr2=0x7f11440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 -- 172.21.15.46:0/2013170267 shutdown_connections 2024-09-12T03:07:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.764+0000 7f1149903640 1 -- 172.21.15.46:0/2013170267 wait complete. 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 Processor -- start 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 -- start start 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 0x7f114419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1144114120 con 0x7f11441009d0 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1144113fa0 con 0x7f114410e4f0 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1149903640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11441142a0 con 0x7f1144101380 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48000/0 (socket says 172.21.15.46:48000) 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 -- 172.21.15.46:0/2068413520 learned_addr learned my addr 172.21.15.46:0/2068413520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f11427fc640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f11437fe640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 0x7f114419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 -- 172.21.15.46:0/2068413520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 msgr2=0x7f114419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 -- 172.21.15.46:0/2068413520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 msgr2=0x7f114419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 0x7f114419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f1142ffd640 1 -- 172.21.15.46:0/2068413520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114419f650 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.765+0000 7f11427fc640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1142ffd640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1134005940 tx=0x7f11340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113403f030 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11340044c0 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114419f8e0 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134035070 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.766+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11441a7220 con 0x7f11441009d0 2024-09-12T03:07:31.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.767+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114406a7d0 con 0x7f11441009d0 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.768+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1134042030 con 0x7f11441009d0 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.769+0000 7f1148901640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 0x7f11300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.769+0000 7f11427fc640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 0x7f11300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.769+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7f11340efbb0 con 0x7f11441009d0 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.770+0000 7f11427fc640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 0x7f11300a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f114410e980 tx=0x7f112c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.774+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11340bc3c0 con 0x7f11441009d0 2024-09-12T03:07:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.961+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1144109680 con 0x7f11441009d0 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.961+0000 7f1148901640 1 -- 172.21.15.46:0/2068413520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v118) ==== 81+0+943 (secure 0 0 0) 0x7f11340bbcc0 con 0x7f11441009d0 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/328833784 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3748867549 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/829485206 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2351168704 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:31.962 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2520496515 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1895867717 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1022575768 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/791127312 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2423476506 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3748867549 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4279295285 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2936988098 2024-09-13T02:40:36.157891+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4229481015 2024-09-13T02:40:11.855373+0000 2024-09-12T03:07:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2008665137 2024-09-13T02:50:10.431130+0000 2024-09-12T03:07:31.964 INFO:teuthology.orchestra.run.smithi046.stderr:listed 16 entries 2024-09-12T03:07:31.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 msgr2=0x7f11300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:31.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 0x7f11300a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f114410e980 tx=0x7f112c005f90 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 msgr2=0x7f114419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1134005940 tx=0x7f11340040c0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 shutdown_connections 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f11300a2d70 0x7f11300a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114410e4f0 0x7f114419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144101380 0x7f114419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.963+0000 7f1149903640 1 --2- 172.21.15.46:0/2068413520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11441009d0 0x7f114419a6a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.964+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 >> 172.21.15.46:0/2068413520 conn(0x7f11440fc7d0 msgr2=0x7f11440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.964+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 shutdown_connections 2024-09-12T03:07:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:31.964+0000 7f1149903640 1 -- 172.21.15.46:0/2068413520 wait complete. 2024-09-12T03:07:32.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:32 smithi046 ceph-mon[30951]: pgmap v548: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:32.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:32 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2068413520' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:32.658 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-09-12T03:07:32.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:07:32.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T03:07:32.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:07:32.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[85429]: fuse finished with error 0 and tester_r 0 2024-09-12T03:07:32.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:32 smithi145 ceph-mon[34674]: pgmap v548: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:32.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:32 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2068413520' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:32.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:32 smithi196 ceph-mon[34587]: pgmap v548: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:07:32.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:32 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2068413520' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:07:34.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:34 smithi046 ceph-mon[30951]: pgmap v549: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:07:34.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:34 smithi145 ceph-mon[34674]: pgmap v549: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:07:34.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:34 smithi196 ceph-mon[34587]: pgmap v549: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:07:36.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:36 smithi046 ceph-mon[30951]: pgmap v550: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:36.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:36 smithi145 ceph-mon[34674]: pgmap v550: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:36.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:36 smithi196 ceph-mon[34587]: pgmap v550: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:38.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:38 smithi046 ceph-mon[30951]: pgmap v551: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T03:07:38.803 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T03:07:38.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:07:38.804 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:07:38.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:07:38.825 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-12T03:07:38.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:38 smithi145 ceph-mon[34674]: pgmap v551: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T03:07:38.907 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:07:38.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:07:38.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:07:38.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:38 smithi196 ceph-mon[34587]: pgmap v551: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T03:07:38.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:07:38.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-12T03:07:39.073 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:07:39.084 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:07:39.084 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:07:39.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:39.351 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- 172.21.15.46:0/643582181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0106f10 msgr2=0x7fc5b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 --2- 172.21.15.46:0/643582181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0106f10 0x7fc5b010ea90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009ca0 tx=0x7fc5a802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- 172.21.15.46:0/643582181 shutdown_connections 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 --2- 172.21.15.46:0/643582181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0106f10 0x7fc5b010ea90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 --2- 172.21.15.46:0/643582181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0102790 0x7fc5b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 --2- 172.21.15.46:0/643582181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 0x7fc5b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- 172.21.15.46:0/643582181 >> 172.21.15.46:0/643582181 conn(0x7fc5b00fd8c0 msgr2=0x7fc5b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- 172.21.15.46:0/643582181 shutdown_connections 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- 172.21.15.46:0/643582181 wait complete. 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 Processor -- start 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.906+0000 7fc5b7647640 1 -- start start 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 0x7fc5b01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0106f10 0x7fc5b019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0111e20 con 0x7fc5b0102790 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0111ca0 con 0x7fc5b0106f10 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b7647640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0111fa0 con 0x7fc5b0101de0 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60864/0 (socket says 172.21.15.46:60864) 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 -- 172.21.15.46:0/2027761300 learned_addr learned my addr 172.21.15.46:0/2027761300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b53e3640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 0x7fc5b01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b5be4640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0106f10 0x7fc5b019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 -- 172.21.15.46:0/2027761300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 msgr2=0x7fc5b01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 0x7fc5b01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 -- 172.21.15.46:0/2027761300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0106f10 msgr2=0x7fc5b019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0106f10 0x7fc5b019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 -- 172.21.15.46:0/2027761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b019d290 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5b4be2640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c890 tx=0x7fc5a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.907+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004220 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.908+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b019d580 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.908+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5a00043c0 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.908+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004a00 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.908+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a4e60 con 0x7fc5b0102790 2024-09-12T03:07:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.909+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b01021c0 con 0x7fc5b0102790 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.910+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc5a0004ba0 con 0x7fc5b0102790 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.911+0000 7fc5a67fc640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 0x7fc59c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.911+0000 7fc5b53e3640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 0x7fc59c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.911+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7fc5a00cf840 con 0x7fc5b0102790 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.911+0000 7fc5b53e3640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 0x7fc59c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00bbe0 tx=0x7fc5ac00b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:39.914+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5a0014030 con 0x7fc5b0102790 2024-09-12T03:07:40.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.125+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5b010a150 con 0x7fc5b0102790 2024-09-12T03:07:40.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.126+0000 7fc5a67fc640 1 -- 172.21.15.46:0/2027761300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11931 (secure 0 0 0) 0x7fc5a009ba00 con 0x7fc5b0102790 2024-09-12T03:07:40.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T03:07:40.127 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"btime":"2024-09-12T03:06:16:949408+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T03:06:16.476954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T03:07:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 msgr2=0x7fc59c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 0x7fc59c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00bbe0 tx=0x7fc5ac00b390 comp rx=0 tx=0).stop 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 msgr2=0x7fc5b0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c890 tx=0x7fc5a000cd50 comp rx=0 tx=0).stop 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 shutdown_connections 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fc59c0a2cf0 0x7fc59c0a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5b0106f10 0x7fc5b019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:40.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b0102790 0x7fc5b0198820 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 --2- 172.21.15.46:0/2027761300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5b0101de0 0x7fc5b01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 >> 172.21.15.46:0/2027761300 conn(0x7fc5b00fd8c0 msgr2=0x7fc5b00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 shutdown_connections 2024-09-12T03:07:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:40.130+0000 7fc5b7647640 1 -- 172.21.15.46:0/2027761300 wait complete. 2024-09-12T03:07:40.754 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:40 smithi046 ceph-mon[30951]: pgmap v552: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:40.755 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:40 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/2027761300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:40.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:40 smithi145 ceph-mon[34674]: pgmap v552: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:40.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:40 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/2027761300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:40.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:40 smithi196 ceph-mon[34587]: pgmap v552: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:40.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:40 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/2027761300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 -- 172.21.15.46:0/376893434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc106a80 msgr2=0x7f1fbc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 --2- 172.21.15.46:0/376893434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc106a80 0x7f1fbc106e60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009930 tx=0x7f1fac02a420 comp rx=0 tx=0).stop 2024-09-12T03:07:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 -- 172.21.15.46:0/376893434 shutdown_connections 2024-09-12T03:07:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 --2- 172.21.15.46:0/376893434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc101420 0x7f1fbc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 --2- 172.21.15.46:0/376893434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc100a80 0x7f1fbc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 --2- 172.21.15.46:0/376893434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc106a80 0x7f1fbc106e60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 -- 172.21.15.46:0/376893434 >> 172.21.15.46:0/376893434 conn(0x7f1fbc0fc7a0 msgr2=0x7f1fbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 -- 172.21.15.46:0/376893434 shutdown_connections 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.534+0000 7f1fc2742640 1 -- 172.21.15.46:0/376893434 wait complete. 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 Processor -- start 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 -- start start 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc100a80 0x7f1fbc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 0x7f1fbc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc110d70 con 0x7f1fbc101420 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc110bf0 con 0x7f1fbc106a80 2024-09-12T03:07:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc2742640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc110ef0 con 0x7f1fbc100a80 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60888/0 (socket says 172.21.15.46:60888) 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 -- 172.21.15.46:0/208586287 learned_addr learned my addr 172.21.15.46:0/208586287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fbbfff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc100a80 0x7f1fbc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fc0cdf640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 0x7f1fbc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 -- 172.21.15.46:0/208586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc100a80 msgr2=0x7f1fbc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc100a80 0x7f1fbc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 -- 172.21.15.46:0/208586287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 msgr2=0x7f1fbc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.535+0000 7f1fb3fff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 0x7f1fbc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fb3fff640 1 -- 172.21.15.46:0/208586287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc19f5a0 con 0x7f1fbc101420 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fc0cdf640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 0x7f1fbc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fb3fff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400b4a0 tx=0x7f1fa400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4014070 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa4010ac0 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc19f890 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4017c30 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.536+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1a7170 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.537+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc100ee0 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.538+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1fa4017070 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.539+0000 7f1fb9ffb640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 0x7f1fa80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.539+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7f1fa40cede0 con 0x7f1fbc101420 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.539+0000 7f1fbbfff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 0x7f1fa80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.539+0000 7f1fbbfff640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 0x7f1fa80a5230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac032c80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.543+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fa409b760 con 0x7f1fbc101420 2024-09-12T03:07:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.753+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fbc1a7500 con 0x7f1fbc101420 2024-09-12T03:07:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.754+0000 7f1fb9ffb640 1 -- 172.21.15.46:0/208586287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11931 (secure 0 0 0) 0x7f1fa409b060 con 0x7f1fbc101420 2024-09-12T03:07:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T03:07:41.755 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"btime":"2024-09-12T03:06:16:949408+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2542241089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2542241089},{"type":"v1","addr":"172.21.15.46:6835","nonce":2542241089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4040861117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4040861117},{"type":"v1","addr":"172.21.15.46:6837","nonce":4040861117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2433411125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2433411125},{"type":"v1","addr":"172.21.15.46:6839","nonce":2433411125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1897348158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1897348158},{"type":"v1","addr":"172.21.15.46:6841","nonce":1897348158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1820698401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1820698401},{"type":"v1","addr":"172.21.15.145:6833","nonce":1820698401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/198433993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":198433993},{"type":"v1","addr":"172.21.15.145:6835","nonce":198433993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:47:41.799879+0000","modified":"2024-09-12T03:06:16.476954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24476,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1048929159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1048929159},{"type":"v1","addr":"172.21.15.196:6833","nonce":1048929159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2556040092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2556040092},{"type":"v1","addr":"172.21.15.145:6837","nonce":2556040092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/3561512892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3561512892},{"type":"v1","addr":"172.21.15.196:6835","nonce":3561512892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6837/756659635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":756659635},{"type":"v1","addr":"172.21.15.196:6837","nonce":756659635}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.145:6839/1761070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":1761070564},{"type":"v1","addr":"172.21.15.145:6839","nonce":1761070564}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6839/1262634008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1262634008},{"type":"v1","addr":"172.21.15.196:6839","nonce":1262634008}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24476,24460]},"id":1}]} 2024-09-12T03:07:41.755 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 msgr2=0x7f1fa80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 0x7f1fa80a5230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac032c80 comp rx=0 tx=0).stop 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 msgr2=0x7f1fbc19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400b4a0 tx=0x7f1fa400b960 comp rx=0 tx=0).stop 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 shutdown_connections 2024-09-12T03:07:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f1fa80a2d70 0x7f1fa80a5230 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc106a80 0x7f1fbc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc101420 0x7f1fbc19ab30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 --2- 172.21.15.46:0/208586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fbc100a80 0x7f1fbc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 >> 172.21.15.46:0/208586287 conn(0x7f1fbc0fc7a0 msgr2=0x7f1fbc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 shutdown_connections 2024-09-12T03:07:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:41.756+0000 7f1fc2742640 1 -- 172.21.15.46:0/208586287 wait complete. 2024-09-12T03:07:42.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:07:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:42.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:42 smithi046 ceph-mon[30951]: pgmap v553: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:42.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:42 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/208586287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:42 smithi145 ceph-mon[34674]: pgmap v553: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:42.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:42 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/208586287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:42.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:42 smithi196 ceph-mon[34587]: pgmap v553: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:42.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:42 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/208586287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 -- 172.21.15.46:0/580995195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0103780 msgr2=0x7f3fc0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/580995195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0103780 0x7f3fc0107bc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009930 tx=0x7f3fb002a420 comp rx=0 tx=0).stop 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 -- 172.21.15.46:0/580995195 shutdown_connections 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/580995195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0108300 0x7f3fc010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/580995195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0103780 0x7f3fc0107bc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/580995195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0102dd0 0x7f3fc01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 -- 172.21.15.46:0/580995195 >> 172.21.15.46:0/580995195 conn(0x7f3fc00fc7d0 msgr2=0x7f3fc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 -- 172.21.15.46:0/580995195 shutdown_connections 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 -- 172.21.15.46:0/580995195 wait complete. 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.275+0000 7f3fc84ba640 1 Processor -- start 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 -- start start 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103780 0x7f3fc0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 0x7f3fc019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0111f20 con 0x7f3fc0102dd0 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0111da0 con 0x7f3fc0108300 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc84ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01120a0 con 0x7f3fc0103780 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60906/0 (socket says 172.21.15.46:60906) 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 -- 172.21.15.46:0/3728058357 learned_addr learned my addr 172.21.15.46:0/3728058357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6a57640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 0x7f3fc019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc5a55640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103780 0x7f3fc0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 -- 172.21.15.46:0/3728058357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103780 msgr2=0x7f3fc0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103780 0x7f3fc0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 -- 172.21.15.46:0/3728058357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 msgr2=0x7f3fc019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 0x7f3fc019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 -- 172.21.15.46:0/3728058357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc019d4d0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6a57640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 0x7f3fc019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:43.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.276+0000 7f3fc6256640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d950 tx=0x7f3fbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.277+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc017070 con 0x7f3fc0102dd0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.277+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fbc009c30 con 0x7f3fc0102dd0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.277+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc019d7c0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.277+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00f620 con 0x7f3fc0102dd0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.277+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01a50a0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.278+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc01031b0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.279+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3fbc0109a0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.280+0000 7f3fb77fe640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 0x7f3fac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.280+0000 7f3fc5a55640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 0x7f3fac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.280+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7f3fbc0cf4c0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.280+0000 7f3fc5a55640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 0x7f3fac0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009900 tx=0x7f3fb0000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.283+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fbc009de0 con 0x7f3fc0102dd0 2024-09-12T03:07:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.459+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fc010a290 con 0x7f3fc0102dd0 2024-09-12T03:07:43.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.460+0000 7f3fb77fe640 1 -- 172.21.15.46:0/3728058357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+25268 (secure 0 0 0) 0x7f3fbc09bd30 con 0x7f3fc0102dd0 2024-09-12T03:07:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-12T03:07:43.462 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"fsid":"2691fc0c-70b0-11ef-bcea-c7b262605968","created":"2024-09-12T02:39:49.303412+0000","modified":"2024-09-12T03:07:20.394648+0000","last_up_change":"2024-09-12T02:46:48.399399+0000","last_in_change":"2024-09-12T02:46:24.608692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:42:56.661399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:47:37.749578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:47:40.084230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T02:47:46.560164+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aec0a35b-0993-4783-ae91-001115bfc027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6803","nonce":836973982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6805","nonce":836973982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6809","nonce":836973982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":836973982},{"type":"v1","addr":"172.21.15.46:6807","nonce":836973982}]},"public_addr":"172.21.15.46:6803/836973982","cluster_addr":"172.21.15.46:6805/836973982","heartbeat_back_addr":"172.21.15.46:6809/836973982","heartbeat_front_addr":"172.21.15.46:6807/836973982","state":["exists","up"]},{"osd":1,"uuid":"701e099d-ad1c-45cd-b9e3-8164c0daa034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6801","nonce":1956814217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6803","nonce":1956814217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6807","nonce":1956814217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1956814217},{"type":"v1","addr":"172.21.15.145:6805","nonce":1956814217}]},"public_addr":"172.21.15.145:6801/1956814217","cluster_addr":"172.21.15.145:6803/1956814217","heartbeat_back_addr":"172.21.15.145:6807/1956814217","heartbeat_front_addr":"172.21.15.145:6805/1956814217","state":["exists","up"]},{"osd":2,"uuid":"ab049462-5f42-4b64-8536-edca415271ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6801","nonce":3433225191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6803","nonce":3433225191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6807","nonce":3433225191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3433225191},{"type":"v1","addr":"172.21.15.196:6805","nonce":3433225191}]},"public_addr":"172.21.15.196:6801/3433225191","cluster_addr":"172.21.15.196:6803/3433225191","heartbeat_back_addr":"172.21.15.196:6807/3433225191","heartbeat_front_addr":"172.21.15.196:6805/3433225191","state":["exists","up"]},{"osd":3,"uuid":"6ad0adf4-ee5f-4d68-8b92-27397454b3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6811","nonce":3782480680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6813","nonce":3782480680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6817","nonce":3782480680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":3782480680},{"type":"v1","addr":"172.21.15.46:6815","nonce":3782480680}]},"public_addr":"172.21.15.46:6811/3782480680","cluster_addr":"172.21.15.46:6813/3782480680","heartbeat_back_addr":"172.21.15.46:6817/3782480680","heartbeat_front_addr":"172.21.15.46:6815/3782480680","state":["exists","up"]},{"osd":4,"uuid":"62730f75-0e03-48fa-9cd3-d04a9e1d0046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6809","nonce":473009837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6811","nonce":473009837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6815","nonce":473009837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":473009837},{"type":"v1","addr":"172.21.15.145:6813","nonce":473009837}]},"public_addr":"172.21.15.145:6809/473009837","cluster_addr":"172.21.15.145:6811/473009837","heartbeat_back_addr":"172.21.15.145:6815/473009837","heartbeat_front_addr":"172.21.15.145:6813/473009837","state":["exists","up"]},{"osd":5,"uuid":"50ec1203-dfd3-402e-ae2f-e1c914c74c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6809","nonce":859658761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6811","nonce":859658761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6815","nonce":859658761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":859658761},{"type":"v1","addr":"172.21.15.196:6813","nonce":859658761}]},"public_addr":"172.21.15.196:6809/859658761","cluster_addr":"172.21.15.196:6811/859658761","heartbeat_back_addr":"172.21.15.196:6815/859658761","heartbeat_front_addr":"172.21.15.196:6813/859658761","state":["exists","up"]},{"osd":6,"uuid":"499ded47-5f1d-433f-8a32-c75b6b64106a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6819","nonce":2825700382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6821","nonce":2825700382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6825","nonce":2825700382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2825700382},{"type":"v1","addr":"172.21.15.46:6823","nonce":2825700382}]},"public_addr":"172.21.15.46:6819/2825700382","cluster_addr":"172.21.15.46:6821/2825700382","heartbeat_back_addr":"172.21.15.46:6825/2825700382","heartbeat_front_addr":"172.21.15.46:6823/2825700382","state":["exists","up"]},{"osd":7,"uuid":"7de3c410-22b1-47ea-94ef-93b756a84fc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6817","nonce":1917623570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6819","nonce":1917623570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6823","nonce":1917623570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1917623570},{"type":"v1","addr":"172.21.15.145:6821","nonce":1917623570}]},"public_addr":"172.21.15.145:6817/1917623570","cluster_addr":"172.21.15.145:6819/1917623570","heartbeat_back_addr":"172.21.15.145:6823/1917623570","heartbeat_front_addr":"172.21.15.145:6821/1917623570","state":["exists","up"]},{"osd":8,"uuid":"040d8985-a30f-4bdd-a469-0be6e29b98a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6817","nonce":724845953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6819","nonce":724845953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6823","nonce":724845953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":724845953},{"type":"v1","addr":"172.21.15.196:6821","nonce":724845953}]},"public_addr":"172.21.15.196:6817/724845953","cluster_addr":"172.21.15.196:6819/724845953","heartbeat_back_addr":"172.21.15.196:6823/724845953","heartbeat_front_addr":"172.21.15.196:6821/724845953","state":["exists","up"]},{"osd":9,"uuid":"5445403e-05c9-4020-b773-336395f26213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6827","nonce":182095023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6829","nonce":182095023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6833","nonce":182095023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":182095023},{"type":"v1","addr":"172.21.15.46:6831","nonce":182095023}]},"public_addr":"172.21.15.46:6827/182095023","cluster_addr":"172.21.15.46:6829/182095023","heartbeat_back_addr":"172.21.15.46:6833/182095023","heartbeat_front_addr":"172.21.15.46:6831/182095023","state":["exists","up"]},{"osd":10,"uuid":"f5fbcc26-7505-434d-b947-79310be95234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6825","nonce":140848472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6827","nonce":140848472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6831","nonce":140848472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":140848472},{"type":"v1","addr":"172.21.15.145:6829","nonce":140848472}]},"public_addr":"172.21.15.145:6825/140848472","cluster_addr":"172.21.15.145:6827/140848472","heartbeat_back_addr":"172.21.15.145:6831/140848472","heartbeat_front_addr":"172.21.15.145:6829/140848472","state":["exists","up"]},{"osd":11,"uuid":"f2a6c3d9-b9e3-4985-b4b1-e1ac67272126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6825","nonce":3261621208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6827","nonce":3261621208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6831","nonce":3261621208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3261621208},{"type":"v1","addr":"172.21.15.196:6829","nonce":3261621208}]},"public_addr":"172.21.15.196:6825/3261621208","cluster_addr":"172.21.15.196:6827/3261621208","heartbeat_back_addr":"172.21.15.196:6831/3261621208","heartbeat_front_addr":"172.21.15.196:6829/3261621208","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:00.917582+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:26.567982+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:42:52.544676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:17.855379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:43:44.365280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:10.330090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:44:36.458683+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:01.728002+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:27.377723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:45:54.488657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:20.228619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:46:46.516113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/328833784":"2024-09-13T02:50:10.431130+0000","172.21.15.46:6801/3748867549":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/829485206":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2351168704":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2520496515":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/1895867717":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/1022575768":"2024-09-13T02:50:10.431130+0000","172.21.15.46:0/791127312":"2024-09-13T02:40:11.855373+0000","172.21.15.46:6800/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2423476506":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6800/3748867549":"2024-09-13T02:50:10.431130+0000","172.21.15.46:6801/4279295285":"2024-09-13T02:40:36.157891+0000","172.21.15.46:0/2936988098":"2024-09-13T02:40:36.157891+0000","172.21.15.46:6801/4229481015":"2024-09-13T02:40:11.855373+0000","172.21.15.46:0/2008665137":"2024-09-13T02:50:10.431130+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:07:43.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.462+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 msgr2=0x7f3fac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:43.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.462+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 0x7f3fac0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009900 tx=0x7f3fb0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.462+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 msgr2=0x7f3fc0198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d950 tx=0x7f3fbc00de10 comp rx=0 tx=0).stop 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 shutdown_connections 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f3fac0a2cf0 0x7f3fac0a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fc0108300 0x7f3fc019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103780 0x7f3fc0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 --2- 172.21.15.46:0/3728058357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc0102dd0 0x7f3fc0198520 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 >> 172.21.15.46:0/3728058357 conn(0x7f3fc00fc7d0 msgr2=0x7f3fc0105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 shutdown_connections 2024-09-12T03:07:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:43.463+0000 7f3fc84ba640 1 -- 172.21.15.46:0/3728058357 wait complete. 2024-09-12T03:07:43.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:43 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3728058357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:07:43.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:43 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3728058357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:07:43.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:43 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3728058357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:07:44.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:07:44.374 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:44.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:44 smithi046 ceph-mon[30951]: pgmap v554: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:07:44.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:44 smithi145 ceph-mon[34674]: pgmap v554: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- 172.21.15.46:0/2192367261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00100950 msgr2=0x7efe00100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 --2- 172.21.15.46:0/2192367261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00100950 0x7efe00100d30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009900 tx=0x7efdf802a410 comp rx=0 tx=0).stop 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- 172.21.15.46:0/2192367261 shutdown_connections 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 --2- 172.21.15.46:0/2192367261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe0010e500 0x7efe001108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 --2- 172.21.15.46:0/2192367261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 --2- 172.21.15.46:0/2192367261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe00100950 0x7efe00100d30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- 172.21.15.46:0/2192367261 >> 172.21.15.46:0/2192367261 conn(0x7efe000fc7a0 msgr2=0x7efe000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- 172.21.15.46:0/2192367261 shutdown_connections 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- 172.21.15.46:0/2192367261 wait complete. 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 Processor -- start 2024-09-12T03:07:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.867+0000 7efe05745640 1 -- start start 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe00100950 0x7efe0019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe001140f0 con 0x7efe0010e500 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe00113f70 con 0x7efe00101300 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efe05745640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe00114270 con 0x7efe00100950 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55352/0 (socket says 172.21.15.46:55352) 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 -- 172.21.15.46:0/228786165 learned_addr learned my addr 172.21.15.46:0/228786165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdfeffd640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe00100950 0x7efe0019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 -- 172.21.15.46:0/228786165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe00100950 msgr2=0x7efe0019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe00100950 0x7efe0019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 -- 172.21.15.46:0/228786165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 msgr2=0x7efe0019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdfe7fc640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 -- 172.21.15.46:0/228786165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0019f5a0 con 0x7efe0010e500 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdfe7fc640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.868+0000 7efdff7fe640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b9e0 tx=0x7efdf000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.869+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0019070 con 0x7efe0010e500 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.869+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf0014440 con 0x7efe0010e500 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.869+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0019f890 con 0x7efe0010e500 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.869+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00125a0 con 0x7efe0010e500 2024-09-12T03:07:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.869+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a7170 con 0x7efe0010e500 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.870+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00100d30 con 0x7efe0010e500 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.871+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efdf001c070 con 0x7efe0010e500 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.872+0000 7efdf7fff640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 0x7efdec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.872+0000 7efdfeffd640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 0x7efdec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.872+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9467+0+0 (secure 0 0 0) 0x7efdf00cebe0 con 0x7efe0010e500 2024-09-12T03:07:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.872+0000 7efdfeffd640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 0x7efdec0a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdf8005ab0 tx=0x7efdf8005a00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:44.876+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdf009aeb0 con 0x7efe0010e500 2024-09-12T03:07:44.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:44 smithi196 ceph-mon[34587]: pgmap v554: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:07:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.088+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe00109100 con 0x7efe0010e500 2024-09-12T03:07:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.090+0000 7efdf7fff640 1 -- 172.21.15.46:0/228786165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7efdf009f1b0 con 0x7efe0010e500 2024-09-12T03:07:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.093+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 msgr2=0x7efdec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.093+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 0x7efdec0a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdf8005ab0 tx=0x7efdf8005a00 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.093+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 msgr2=0x7efe0019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.093+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b9e0 tx=0x7efdf000bea0 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.093+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 shutdown_connections 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7efdec0a2cf0 0x7efdec0a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0010e500 0x7efe0019eec0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe00101300 0x7efe0019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 --2- 172.21.15.46:0/228786165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe00100950 0x7efe0019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 >> 172.21.15.46:0/228786165 conn(0x7efe000fc7a0 msgr2=0x7efe000feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:45.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 shutdown_connections 2024-09-12T03:07:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:45.094+0000 7efe05745640 1 -- 172.21.15.46:0/228786165 wait complete. 2024-09-12T03:07:45.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[63793]: -- 172.21.15.196:0/1140553442 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574c23cf860 con 0x5574c23ad400 2024-09-12T03:07:45.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: starting mds.l at 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c[64394]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c[64394]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[64398]: -- 172.21.15.196:0/1452614227 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a09ac2a680 con 0x55a09abff400 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[64398]: -- 172.21.15.196:0/1452614227 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a09ac21860 con 0x55a09abff400 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c[64394]: starting mds.c at 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.471 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[64094]: -- 172.21.15.196:0/2665308168 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ccfb4e2680 con 0x55ccfb4b7000 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: starting mds.f at 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[64711]: -- 172.21.15.196:0/1837865496 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5574116d6680 con 0x5574116ab400 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mds[64711]: -- 172.21.15.196:0/1837865496 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574116cdc20 con 0x5574116ab400 2024-09-12T03:07:45.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: starting mds.i at 2024-09-12T03:07:45.523 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a[30947]: 2024-09-12T03:07:45.089+0000 7fd86ae45640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:07:45.525 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.525 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mds[64012]: -- 172.21.15.145:0/2770838059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c75138b860 con 0x55c751368800 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: starting mds.h at 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: ignoring --setuser ceph since I am not root 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: ignoring --setgroup ceph since I am not root 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mds[64317]: -- 172.21.15.145:0/4267207002 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f537d1a680 con 0x55f537cee800 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mds[64317]: -- 172.21.15.145:0/4267207002 <== mon.2 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f537d11860 con 0x55f537cee800 2024-09-12T03:07:45.526 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: starting mds.b at 2024-09-12T03:07:45.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: pgmap v555: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:45.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:07:45.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:07:45.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:07:45.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:07:45.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T03:07:45.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:45 smithi046 ceph-mon[30951]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: pgmap v555: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T03:07:45.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:45 smithi145 ceph-mon[34674]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:07:45.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:07:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: pgmap v555: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:07:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:07:45.970 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:07:45.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/228786165' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:07:45.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T03:07:45.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:45 smithi196 ceph-mon[34587]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:07:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:46.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 -- 172.21.15.46:0/1485045506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118100a80 msgr2=0x7fa118100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/1485045506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118100a80 0x7fa118100ee0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa108009930 tx=0x7fa10802a420 comp rx=0 tx=0).stop 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 -- 172.21.15.46:0/1485045506 shutdown_connections 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/1485045506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 0x7fa11810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/1485045506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118100a80 0x7fa118100ee0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/1485045506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118106a80 0x7fa118106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 -- 172.21.15.46:0/1485045506 >> 172.21.15.46:0/1485045506 conn(0x7fa1180fc7a0 msgr2=0x7fa1180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 -- 172.21.15.46:0/1485045506 shutdown_connections 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.639+0000 7fa11f4ca640 1 -- 172.21.15.46:0/1485045506 wait complete. 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 Processor -- start 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 -- start start 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118100a80 0x7fa118199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 0x7fa118199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa118110d50 con 0x7fa118106a80 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa118110bd0 con 0x7fa118101420 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11f4ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa118110ed0 con 0x7fa118100a80 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.640+0000 7fa11da67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55366/0 (socket says 172.21.15.46:55366) 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 -- 172.21.15.46:0/3273206404 learned_addr learned my addr 172.21.15.46:0/3273206404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11ca65640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 0x7fa118199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11d266640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118100a80 0x7fa118199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 -- 172.21.15.46:0/3273206404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118100a80 msgr2=0x7fa118199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118100a80 0x7fa118199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 -- 172.21.15.46:0/3273206404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 msgr2=0x7fa118199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 0x7fa118199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 -- 172.21.15.46:0/3273206404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1181a47c0 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11da67640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b970 tx=0x7fa11000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110019070 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa110014440 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1181a4990 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100075d0 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.641+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1181a4e50 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.642+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa118100ee0 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.644+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa11001c070 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.644+0000 7fa10e7fc640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 0x7fa1040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.645+0000 7fa11d266640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 0x7fa1040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.645+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9637+0+0 (secure 0 0 0) 0x7fa1100ced80 con 0x7fa118106a80 2024-09-12T03:07:46.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.645+0000 7fa11d266640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 0x7fa1040a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa114005e10 tx=0x7fa114005da0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.649+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa11009b000 con 0x7fa118106a80 2024-09-12T03:07:46.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:46.862+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa118109010 con 0x7fa118106a80 2024-09-12T03:07:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.097+0000 7fa10e7fc640 1 -- 172.21.15.46:0/3273206404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7fa11009f300 con 0x7fa118106a80 2024-09-12T03:07:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 msgr2=0x7fa1040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 0x7fa1040a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa114005e10 tx=0x7fa114005da0 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 msgr2=0x7fa1181a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b970 tx=0x7fa11000be30 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 shutdown_connections 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7fa1040a2cf0 0x7fa1040a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa118106a80 0x7fa1181a4280 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa118101420 0x7fa118199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 --2- 172.21.15.46:0/3273206404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa118100a80 0x7fa118199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.100+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 >> 172.21.15.46:0/3273206404 conn(0x7fa1180fc7a0 msgr2=0x7fa1180fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.101+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 shutdown_connections 2024-09-12T03:07:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:47.101+0000 7fa11f4ca640 1 -- 172.21.15.46:0/3273206404 wait complete. 2024-09-12T03:07:47.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6832/4224051587,v1:172.21.15.196:6836/4224051587] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6834/1624030601,v1:172.21.15.196:6838/1624030601] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6833/1019282784,v1:172.21.15.196:6837/1019282784] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.196:6835/3759402421,v1:172.21.15.196:6839/3759402421] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6836/2167448681,v1:172.21.15.145:6838/2167448681] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: mds.? [v2:172.21.15.145:6837/3332568056,v1:172.21.15.145:6839/3332568056] up:boot 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:07:47.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:47 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:07:47.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6832/4224051587,v1:172.21.15.196:6836/4224051587] up:boot 2024-09-12T03:07:47.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6834/1624030601,v1:172.21.15.196:6838/1624030601] up:boot 2024-09-12T03:07:47.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6833/1019282784,v1:172.21.15.196:6837/1019282784] up:boot 2024-09-12T03:07:47.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.196:6835/3759402421,v1:172.21.15.196:6839/3759402421] up:boot 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6836/2167448681,v1:172.21.15.145:6838/2167448681] up:boot 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: mds.? [v2:172.21.15.145:6837/3332568056,v1:172.21.15.145:6839/3332568056] up:boot 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:07:47.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:47 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:07:47.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6832/4224051587,v1:172.21.15.196:6836/4224051587] up:boot 2024-09-12T03:07:47.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6834/1624030601,v1:172.21.15.196:6838/1624030601] up:boot 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6833/1019282784,v1:172.21.15.196:6837/1019282784] up:boot 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.196:6835/3759402421,v1:172.21.15.196:6839/3759402421] up:boot 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6836/2167448681,v1:172.21.15.145:6838/2167448681] up:boot 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: mds.? [v2:172.21.15.145:6837/3332568056,v1:172.21.15.145:6839/3332568056] up:boot 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='mgr.15228 172.21.15.46:0/1346392453' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:07:47.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:47 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:07:47.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:07:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:48.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: pgmap v557: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:07:48.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:07:48.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:07:48.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: Cluster is now healthy 2024-09-12T03:07:48.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:07:48.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:48 smithi145 ceph-mon[34674]: fsmap 12 up:standby 2024-09-12T03:07:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: pgmap v557: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:07:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:07:48.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:07:48.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: Cluster is now healthy 2024-09-12T03:07:48.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:07:48.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:48 smithi196 ceph-mon[34587]: fsmap 12 up:standby 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: pgmap v557: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: Cluster is now healthy 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/3273206404' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:07:48.473 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:48 smithi046 ceph-mon[30951]: fsmap 12 up:standby 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 -- 172.21.15.46:0/3950881038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45981002f0 msgr2=0x7f45981006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 --2- 172.21.15.46:0/3950881038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45981002f0 0x7f45981006d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f458c009930 tx=0x7f458c02a420 comp rx=0 tx=0).stop 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 -- 172.21.15.46:0/3950881038 shutdown_connections 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 --2- 172.21.15.46:0/3950881038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598106f50 0x7f459810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 --2- 172.21.15.46:0/3950881038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4598100ca0 0x7f4598106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 --2- 172.21.15.46:0/3950881038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45981002f0 0x7f45981006d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 -- 172.21.15.46:0/3950881038 >> 172.21.15.46:0/3950881038 conn(0x7f4598077210 msgr2=0x7f45980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 -- 172.21.15.46:0/3950881038 shutdown_connections 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.725+0000 7f459eece640 1 -- 172.21.15.46:0/3950881038 wait complete. 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 Processor -- start 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 -- start start 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45981002f0 0x7f4598101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598100ca0 0x7f45981017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4598111e50 con 0x7f4598106f50 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4598111cd0 con 0x7f45981002f0 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459eece640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4598111fd0 con 0x7f4598100ca0 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.726+0000 7f459d46b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55388/0 (socket says 172.21.15.46:55388) 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f4597fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598100ca0 0x7f45981017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 -- 172.21.15.46:0/726187676 learned_addr learned my addr 172.21.15.46:0/726187676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459cc6a640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45981002f0 0x7f4598101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 -- 172.21.15.46:0/726187676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598100ca0 msgr2=0x7f45981017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598100ca0 0x7f45981017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 -- 172.21.15.46:0/726187676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45981002f0 msgr2=0x7f4598101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45981002f0 0x7f4598101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 -- 172.21.15.46:0/726187676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45981a68d0 con 0x7f4598106f50 2024-09-12T03:07:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459d46b640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f459000eaa0 tx=0x7f459000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590004d20 con 0x7f4598106f50 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45981a6bc0 con 0x7f4598106f50 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4590005020 con 0x7f4598106f50 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.727+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900116d0 con 0x7f4598106f50 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.728+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981a7100 con 0x7f4598106f50 2024-09-12T03:07:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.729+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45981006d0 con 0x7f4598106f50 2024-09-12T03:07:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.730+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f459000caf0 con 0x7f4598106f50 2024-09-12T03:07:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.731+0000 7f4595ffb640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 0x7f45880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.731+0000 7f459cc6a640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 0x7f45880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.731+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9637+0+0 (secure 0 0 0) 0x7f45900cf980 con 0x7f4598106f50 2024-09-12T03:07:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.731+0000 7f459cc6a640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 0x7f45880a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f458c009900 tx=0x7f458c000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:48.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.735+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4590016030 con 0x7f4598106f50 2024-09-12T03:07:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:48.911+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f459802d7c0 con 0x7f4598106f50 2024-09-12T03:07:49.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.106+0000 7f4595ffb640 1 -- 172.21.15.46:0/726187676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f459009c0e0 con 0x7f4598106f50 2024-09-12T03:07:49.106 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.109+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 msgr2=0x7f45880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.109+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 0x7f45880a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f458c009900 tx=0x7f458c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.109+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 msgr2=0x7f4598101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.109+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f459000eaa0 tx=0x7f459000eea0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.109+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 shutdown_connections 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f45880a2cf0 0x7f45880a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4598106f50 0x7f4598101cf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4598100ca0 0x7f45981017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 --2- 172.21.15.46:0/726187676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45981002f0 0x7f4598101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 >> 172.21.15.46:0/726187676 conn(0x7f4598077210 msgr2=0x7f45980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 shutdown_connections 2024-09-12T03:07:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:49.110+0000 7f459eece640 1 -- 172.21.15.46:0/726187676 wait complete. 2024-09-12T03:07:49.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:49 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:49.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:49 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:49.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:49 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:49.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:07:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:50.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:50 smithi145 ceph-mon[34674]: pgmap v558: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:07:50.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:50 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:50.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:50 smithi145 ceph-mon[34674]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T03:07:50.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:50 smithi046 ceph-mon[30951]: pgmap v558: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:07:50.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:50 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:50.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:50 smithi046 ceph-mon[30951]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T03:07:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:50 smithi196 ceph-mon[34587]: pgmap v558: 129 pgs: 129 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:07:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:50 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/726187676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:50.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:50 smithi196 ceph-mon[34587]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T03:07:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 -- 172.21.15.46:0/1762422187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 msgr2=0x7f21501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 --2- 172.21.15.46:0/1762422187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21501021c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f214c009900 tx=0x7f214c02a410 comp rx=0 tx=0).stop 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 -- 172.21.15.46:0/1762422187 shutdown_connections 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 --2- 172.21.15.46:0/1762422187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150106f10 0x7f215010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 --2- 172.21.15.46:0/1762422187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150102790 0x7f21501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 --2- 172.21.15.46:0/1762422187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21501021c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 -- 172.21.15.46:0/1762422187 >> 172.21.15.46:0/1762422187 conn(0x7f21500fd8c0 msgr2=0x7f21500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 -- 172.21.15.46:0/1762422187 shutdown_connections 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.644+0000 7f21587d9640 1 -- 172.21.15.46:0/1762422187 wait complete. 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 Processor -- start 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 -- start start 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150102790 0x7f2150074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150106f10 0x7f2150074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2150111e20 con 0x7f2150101de0 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2150111ca0 con 0x7f2150106f10 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f21587d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2150111fa0 con 0x7f2150102790 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f2156575640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f2156575640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55408/0 (socket says 172.21.15.46:55408) 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f2156575640 1 -- 172.21.15.46:0/397271891 learned_addr learned my addr 172.21.15.46:0/397271891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f2155d74640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150102790 0x7f2150074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.645+0000 7f2156d76640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150106f10 0x7f2150074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 -- 172.21.15.46:0/397271891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150102790 msgr2=0x7f2150074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150102790 0x7f2150074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 -- 172.21.15.46:0/397271891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150106f10 msgr2=0x7f2150074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150106f10 0x7f2150074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 -- 172.21.15.46:0/397271891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21501a6820 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f2156575640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f214c005940 tx=0x7f214c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c03f030 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f214c003aa0 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21501a6ab0 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c035070 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.646+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21501a6fe0 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.647+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21501021c0 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.648+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f214c042030 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.649+0000 7f21477fe640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 0x7f213c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.649+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9141+0+0 (secure 0 0 0) 0x7f214c0effc0 con 0x7f2150101de0 2024-09-12T03:07:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.649+0000 7f2155d74640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 0x7f213c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.650+0000 7f2155d74640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 0x7f213c0a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2150075f80 tx=0x7f214000b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.652+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f214c0bc2c0 con 0x7f2150101de0 2024-09-12T03:07:50.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:50.831+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f215010a0f0 con 0x7f2150101de0 2024-09-12T03:07:51.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.413+0000 7f21477fe640 1 -- 172.21.15.46:0/397271891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7f214c0c05c0 con 0x7f2150101de0 2024-09-12T03:07:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-12T03:07:51.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 msgr2=0x7f213c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:51.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 0x7f213c0a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2150075f80 tx=0x7f214000b390 comp rx=0 tx=0).stop 2024-09-12T03:07:51.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 msgr2=0x7f21500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f214c005940 tx=0x7f214c004270 comp rx=0 tx=0).stop 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 shutdown_connections 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f213c0a2d70 0x7f213c0a5230 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2150106f10 0x7f2150074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2150102790 0x7f2150074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 --2- 172.21.15.46:0/397271891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150101de0 0x7f21500744f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 >> 172.21.15.46:0/397271891 conn(0x7f21500fd8c0 msgr2=0x7f21500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 shutdown_connections 2024-09-12T03:07:51.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:51.418+0000 7f21587d9640 1 -- 172.21.15.46:0/397271891 wait complete. 2024-09-12T03:07:51.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:51 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:51.745 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:51 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:51.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:51 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:52.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T03:07:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:52.589 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:52 smithi046 ceph-mon[30951]: pgmap v560: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:52.589 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:52 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:52.589 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:52 smithi046 ceph-mon[30951]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T03:07:52.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:52 smithi196 ceph-mon[34587]: pgmap v560: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:52.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:52 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:52.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:52 smithi196 ceph-mon[34587]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T03:07:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:52 smithi145 ceph-mon[34674]: pgmap v560: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:52 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/397271891' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:52.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:52 smithi145 ceph-mon[34674]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T03:07:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 -- 172.21.15.46:0/3465718905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed01082a0 msgr2=0x7f6ed010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 --2- 172.21.15.46:0/3465718905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed01082a0 0x7f6ed010ec40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009cd0 tx=0x7f6ec802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 -- 172.21.15.46:0/3465718905 shutdown_connections 2024-09-12T03:07:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 --2- 172.21.15.46:0/3465718905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed01082a0 0x7f6ed010ec40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 --2- 172.21.15.46:0/3465718905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 --2- 172.21.15.46:0/3465718905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed0102d30 0x7f6ed0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 -- 172.21.15.46:0/3465718905 >> 172.21.15.46:0/3465718905 conn(0x7f6ed00fc7d0 msgr2=0x7f6ed00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 -- 172.21.15.46:0/3465718905 shutdown_connections 2024-09-12T03:07:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.941+0000 7f6ed573a640 1 -- 172.21.15.46:0/3465718905 wait complete. 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 Processor -- start 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 -- start start 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 0x7f6ed019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111f50 con 0x7f6ed0102d30 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111dd0 con 0x7f6ed01082a0 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ed573a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01120d0 con 0x7f6ed01036e0 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6eceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6eceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55418/0 (socket says 172.21.15.46:55418) 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6eceffd640 1 -- 172.21.15.46:0/1107807191 learned_addr learned my addr 172.21.15.46:0/1107807191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-12T03:07:52.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ecf7fe640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 0x7f6ed019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.942+0000 7f6ece7fc640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 -- 172.21.15.46:0/1107807191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 msgr2=0x7f6ed0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 -- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 msgr2=0x7f6ed0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 -- 172.21.15.46:0/1107807191 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed019d400 con 0x7f6ed01082a0 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6eceffd640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ece7fc640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ecf7fe640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 0x7f6ed019cd20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802ae00 tx=0x7f6ec8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8004240 con 0x7f6ed01082a0 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed019d690 con 0x7f6ed01082a0 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec80305a0 con 0x7f6ed01082a0 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec803b660 con 0x7f6ed01082a0 2024-09-12T03:07:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.943+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a4fd0 con 0x7f6ed01082a0 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.944+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0103110 con 0x7f6ed01082a0 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.946+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ec8030750 con 0x7f6ed01082a0 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.947+0000 7f6ec7fff640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 0x7f6ebc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.947+0000 7f6eceffd640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 0x7f6ebc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.947+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8582+0+0 (secure 0 0 0) 0x7f6ec802e030 con 0x7f6ed01082a0 2024-09-12T03:07:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.947+0000 7f6eceffd640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 0x7f6ebc0a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005fd0 tx=0x7f6ec000b480 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:52.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:52.950+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ec80bd580 con 0x7f6ed01082a0 2024-09-12T03:07:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.128+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6ed010a2b0 con 0x7f6ed01082a0 2024-09-12T03:07:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.424+0000 7f6ec7fff640 1 -- 172.21.15.46:0/1107807191 <== mon.2 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v122) ==== 178+0+0 (secure 0 0 0) 0x7f6ec80bce80 con 0x7f6ed01082a0 2024-09-12T03:07:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-09-12T03:07:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 msgr2=0x7f6ebc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 0x7f6ebc0a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005fd0 tx=0x7f6ec000b480 comp rx=0 tx=0).stop 2024-09-12T03:07:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 msgr2=0x7f6ed019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 0x7f6ed019cd20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802ae00 tx=0x7f6ec8003ba0 comp rx=0 tx=0).stop 2024-09-12T03:07:53.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 shutdown_connections 2024-09-12T03:07:53.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:6800/1256678118,v1:172.21.15.46:6801/1256678118] conn(0x7f6ebc0a2cf0 0x7f6ebc0a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:53.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ed01082a0 0x7f6ed019cd20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:53.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01036e0 0x7f6ed0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 --2- 172.21.15.46:0/1107807191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed0102d30 0x7f6ed0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.428+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 >> 172.21.15.46:0/1107807191 conn(0x7f6ed00fc7d0 msgr2=0x7f6ed0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.429+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 shutdown_connections 2024-09-12T03:07:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-12T03:07:53.430+0000 7f6ed573a640 1 -- 172.21.15.46:0/1107807191 wait complete. 2024-09-12T03:07:53.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:53 smithi196 ceph-mon[34587]: from='client.? 172.21.15.46:0/1107807191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:53.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:53 smithi196 ceph-mon[34587]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:53.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:53 smithi046 ceph-mon[30951]: from='client.? 172.21.15.46:0/1107807191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:53.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:53 smithi046 ceph-mon[30951]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:53.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:53 smithi145 ceph-mon[34674]: from='client.? 172.21.15.46:0/1107807191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:53.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:53 smithi145 ceph-mon[34674]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:07:54.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:54.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:54.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:54 smithi046 ceph-mon[30951]: pgmap v562: 33 pgs: 33 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:54 smithi046 ceph-mon[30951]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:54.532 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:54 smithi046 ceph-mon[30951]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T03:07:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:54 smithi196 ceph-mon[34587]: pgmap v562: 33 pgs: 33 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:54 smithi196 ceph-mon[34587]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:54.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:07:54 smithi196 ceph-mon[34587]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T03:07:54.791 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:07:54.791 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:07:54.828 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:07:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:54 smithi145 ceph-mon[34674]: pgmap v562: 33 pgs: 33 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:07:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:54 smithi145 ceph-mon[34674]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:07:54.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:54 smithi145 ceph-mon[34674]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T03:07:54.901 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:07:54.933 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:07:54.934 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:07:54.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:07:55.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:54 smithi046 systemd[1]: Stopping Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:07:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:55 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a[30947]: 2024-09-12T03:07:55.289+0000 7fd870650640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:55 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a[30947]: 2024-09-12T03:07:55.289+0000 7fd870650640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:07:55.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:55 smithi046 podman[109238]: 2024-09-12 03:07:55.478963474 +0000 UTC m=+0.315496391 container died 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:07:56.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:56 smithi046 podman[109238]: 2024-09-12 03:07:56.546112943 +0000 UTC m=+1.382645846 container cleanup 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:07:56.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:56 smithi046 bash[109238]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:07:57.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:07:57.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:56 smithi046 podman[109250]: 2024-09-12 03:07:56.898247189 +0000 UTC m=+1.409771275 container remove 3f65ff0b2fe59cfdc2c2fa9d3266b0d92a6ddea3cef67446556fa9a1109bb314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:07:57.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:57 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:07:57.180 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:57 smithi046 systemd[1]: Stopped Ceph mon.a for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T03:07:57.180 INFO:journalctl@ceph.mon.a.smithi046.stdout:Sep 12 03:07:57 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.a.service: Consumed 25.853s CPU time. 2024-09-12T03:07:57.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:57.221 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:07:57.221 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:07:57.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:07:57.610 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:07:57 smithi145 systemd[1]: Stopping Ceph mon.b for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:00.532 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x[31180]: 2024-09-12T03:08:00.118+0000 7fe1eb1e9640 -1 client.15276: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T03:08:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:00 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b[34669]: 2024-09-12T03:08:00.431+0000 7f4eee697640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:08:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:00 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b[34669]: 2024-09-12T03:08:00.431+0000 7f4eee697640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:08:00.860 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:00 smithi145 podman[71316]: 2024-09-12 03:08:00.607606577 +0000 UTC m=+2.244810330 container died be1aba72babc1a1d3b1a7ac9bc34b27117b60efade7af84fe46f309978d74500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:08:07.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:07 smithi145 podman[71316]: 2024-09-12 03:08:07.07817663 +0000 UTC m=+8.715380385 container cleanup be1aba72babc1a1d3b1a7ac9bc34b27117b60efade7af84fe46f309978d74500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:08:07.361 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:07 smithi145 bash[71316]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:08:11.193 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 12 03:08:10 smithi145 podman[71328]: 2024-09-12 03:08:10.91647674 +0000 UTC m=+10.101634206 container remove be1aba72babc1a1d3b1a7ac9bc34b27117b60efade7af84fe46f309978d74500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:08:11.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:08:11.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:11.267 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:08:11.267 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:08:11.267 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:08:11.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:11 smithi196 systemd[1]: Stopping Ceph mon.c for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:14.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:13 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-c[34583]: 2024-09-12T03:08:13.907+0000 7fb078c0e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:08:14.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:13 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-c[34583]: 2024-09-12T03:08:13.907+0000 7fb078c0e640 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-09-12T03:08:14.220 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:14 smithi196 podman[71769]: 2024-09-12 03:08:14.112425731 +0000 UTC m=+2.244786450 container died 58fda31218fb03bfc9e31a9ce88943513b7b248dc828d2ee0f2ebc3b7aceb29f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T03:08:19.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:19 smithi196 podman[71769]: 2024-09-12 03:08:19.419839895 +0000 UTC m=+7.552200615 container cleanup 58fda31218fb03bfc9e31a9ce88943513b7b248dc828d2ee0f2ebc3b7aceb29f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-c, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:08:19.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 12 03:08:19 smithi196 bash[71769]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:08:22.272 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:08:22.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:22.443 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:08:22.444 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:08:22.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:08:22.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:22 smithi046 systemd[1]: Stopping Ceph mgr.x for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:25.033 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:24 smithi046 podman[109359]: 2024-09-12 03:08:24.762099526 +0000 UTC m=+1.526718154 container died f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:08:30.782 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:30 smithi046 podman[109359]: 2024-09-12 03:08:30.327002079 +0000 UTC m=+7.091620706 container cleanup f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:08:30.783 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:30 smithi046 bash[109359]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:08:33.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:08:33.613 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:33 smithi046 podman[109371]: 2024-09-12 03:08:33.343974727 +0000 UTC m=+8.578503719 container remove f9e3e37a643a201d673551655ad75612ef6525c613bb153f9587c153ee6ca383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:08:33.613 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:33 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T03:08:33.613 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:33 smithi046 systemd[1]: Stopped Ceph mgr.x for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T03:08:33.613 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Sep 12 03:08:33 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.x.service: Consumed 1min 2.661s CPU time. 2024-09-12T03:08:33.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:33.690 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:08:33.690 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:08:33.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:08:34.033 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 03:08:33 smithi145 systemd[1]: Stopping Ceph mgr.y for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:34.354 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 03:08:34 smithi145 podman[71445]: 2024-09-12 03:08:34.033405075 +0000 UTC m=+0.144065103 container died d1fa9063c46f56f7cb26fd353bbc7e0d9069ea6afffac2cd53152aebd7e03bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:08:34.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 03:08:34 smithi145 podman[71445]: 2024-09-12 03:08:34.462056093 +0000 UTC m=+0.572716164 container cleanup d1fa9063c46f56f7cb26fd353bbc7e0d9069ea6afffac2cd53152aebd7e03bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:08:34.610 INFO:journalctl@ceph.mgr.y.smithi145.stdout:Sep 12 03:08:34 smithi145 bash[71445]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:08:34.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:08:34.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:34.942 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:08:34.942 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:08:34.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:08:35.220 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 03:08:34 smithi196 systemd[1]: Stopping Ceph mgr.z for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:36.445 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 03:08:36 smithi196 podman[71893]: 2024-09-12 03:08:36.117398359 +0000 UTC m=+0.977185983 container died ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:08:36.720 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 03:08:36 smithi196 podman[71893]: 2024-09-12 03:08:36.552891673 +0000 UTC m=+1.412679294 container cleanup ed3aad2f73a810c87cfbbbc97c5e9cd342f6da632913b92353295c17d344531b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:08:36.720 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 12 03:08:36 smithi196 bash[71893]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:08:36.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:08:36.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:36.955 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:08:36.955 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:08:36.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:08:37.282 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:37 smithi046 systemd[1]: Stopping Ceph osd.0 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:39.282 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T03:08:38.989+0000 7fc7bbcb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:08:39.282 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T03:08:38.989+0000 7fc7bbcb3640 -1 osd.0 122 *** Got signal Terminated *** 2024-09-12T03:08:39.283 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:38 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0[45244]: 2024-09-12T03:08:38.989+0000 7fc7bbcb3640 -1 osd.0 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:08:45.032 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:44 smithi046 podman[109491]: 2024-09-12 03:08:44.690256797 +0000 UTC m=+7.529411784 container died 444219c7655007e2ee91258d4624ac7e7e63fc28e49c326ea47600966dfd5d7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:08:47.282 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109491]: 2024-09-12 03:08:47.007827379 +0000 UTC m=+9.846982365 container cleanup 444219c7655007e2ee91258d4624ac7e7e63fc28e49c326ea47600966dfd5d7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:08:47.282 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 bash[109491]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:08:47.629 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109506]: 2024-09-12 03:08:47.309034348 +0000 UTC m=+2.447130128 container remove 444219c7655007e2ee91258d4624ac7e7e63fc28e49c326ea47600966dfd5d7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T03:08:47.887 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109597]: 2024-09-12 03:08:47.532081522 +0000 UTC m=+0.016468730 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:08:47.887 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109597]: 2024-09-12 03:08:47.643320751 +0000 UTC m=+0.127707947 container create b0dd5a1c8a55d4948ebc3b188c7498256f169b4e401d937f13f472993a4a391f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:08:48.268 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109597]: 2024-09-12 03:08:47.887741962 +0000 UTC m=+0.372129159 container init b0dd5a1c8a55d4948ebc3b188c7498256f169b4e401d937f13f472993a4a391f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:08:48.269 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109597]: 2024-09-12 03:08:47.891250536 +0000 UTC m=+0.375637733 container start b0dd5a1c8a55d4948ebc3b188c7498256f169b4e401d937f13f472993a4a391f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:08:48.269 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:47 smithi046 podman[109597]: 2024-09-12 03:08:47.979770503 +0000 UTC m=+0.464157719 container attach b0dd5a1c8a55d4948ebc3b188c7498256f169b4e401d937f13f472993a4a391f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:08:48.532 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 12 03:08:48 smithi046 podman[109597]: 2024-09-12 03:08:48.268526019 +0000 UTC m=+0.752913231 container died b0dd5a1c8a55d4948ebc3b188c7498256f169b4e401d937f13f472993a4a391f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T03:08:49.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:08:49.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:49.062 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:08:49.062 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:08:49.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:08:49.360 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:49 smithi145 systemd[1]: Stopping Ceph osd.1 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:49.620 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:49 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T03:08:49.396+0000 7f59bc794640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:08:49.620 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:49 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T03:08:49.396+0000 7f59bc794640 -1 osd.1 122 *** Got signal Terminated *** 2024-09-12T03:08:49.620 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:49 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1[39149]: 2024-09-12T03:08:49.396+0000 7f59bc794640 -1 osd.1 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:08:55.592 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:55 smithi145 podman[71572]: 2024-09-12 03:08:55.281584231 +0000 UTC m=+6.027466707 container died ffc4be8979eeea6e8f8fc3f43c915dceba3e2350e306fe7ac6253f7de12cafcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:08:55.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:55 smithi145 podman[71572]: 2024-09-12 03:08:55.69975268 +0000 UTC m=+6.445635156 container cleanup ffc4be8979eeea6e8f8fc3f43c915dceba3e2350e306fe7ac6253f7de12cafcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:08:55.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:55 smithi145 bash[71572]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:08:56.265 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:55 smithi145 podman[71599]: 2024-09-12 03:08:55.951340519 +0000 UTC m=+0.655937518 container remove ffc4be8979eeea6e8f8fc3f43c915dceba3e2350e306fe7ac6253f7de12cafcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:08:56.572 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:56 smithi145 podman[71690]: 2024-09-12 03:08:56.167996368 +0000 UTC m=+0.015526458 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:08:56.572 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:56 smithi145 podman[71690]: 2024-09-12 03:08:56.31065262 +0000 UTC m=+0.158182711 container create 37b1ff1f145c26afa9726fbb858b2f40c3fc837708374ed0fde21c6ec0e1656e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:08:56.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:56 smithi145 podman[71690]: 2024-09-12 03:08:56.571906712 +0000 UTC m=+0.419436803 container init 37b1ff1f145c26afa9726fbb858b2f40c3fc837708374ed0fde21c6ec0e1656e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:08:56.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:56 smithi145 podman[71690]: 2024-09-12 03:08:56.575198758 +0000 UTC m=+0.422728854 container start 37b1ff1f145c26afa9726fbb858b2f40c3fc837708374ed0fde21c6ec0e1656e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:08:56.860 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:56 smithi145 podman[71690]: 2024-09-12 03:08:56.713717625 +0000 UTC m=+0.561247735 container attach 37b1ff1f145c26afa9726fbb858b2f40c3fc837708374ed0fde21c6ec0e1656e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:08:57.360 INFO:journalctl@ceph.osd.1.smithi145.stdout:Sep 12 03:08:57 smithi145 podman[71690]: 2024-09-12 03:08:57.011802737 +0000 UTC m=+0.859332834 container died 37b1ff1f145c26afa9726fbb858b2f40c3fc837708374ed0fde21c6ec0e1656e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:08:57.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:08:57.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:08:57.757 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:08:57.757 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:08:57.757 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:08:58.097 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:08:57 smithi196 systemd[1]: Stopping Ceph osd.2 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:08:58.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:08:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T03:08:58.096+0000 7fef88cf2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:08:58.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:08:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T03:08:58.096+0000 7fef88cf2640 -1 osd.2 122 *** Got signal Terminated *** 2024-09-12T03:08:58.470 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:08:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2[39537]: 2024-09-12T03:08:58.096+0000 7fef88cf2640 -1 osd.2 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:04.220 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:03 smithi196 podman[72016]: 2024-09-12 03:09:03.926150127 +0000 UTC m=+5.979405335 container died ac4a1502d9e932fc0a131dca84947d6cb027470b59160aa626c6222b61a2cade (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T03:09:04.665 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:04 smithi196 podman[72016]: 2024-09-12 03:09:04.35557421 +0000 UTC m=+6.408829415 container cleanup ac4a1502d9e932fc0a131dca84947d6cb027470b59160aa626c6222b61a2cade (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:09:04.665 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:04 smithi196 bash[72016]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:09:04.970 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:04 smithi196 podman[72043]: 2024-09-12 03:09:04.665661889 +0000 UTC m=+0.695536317 container remove ac4a1502d9e932fc0a131dca84947d6cb027470b59160aa626c6222b61a2cade (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T03:09:05.319 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:04.913797935 +0000 UTC m=+0.016039061 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:05.319 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:05.058491857 +0000 UTC m=+0.160733047 container create 4f4bed63695a7ff3dd11f6b8a0fa67a9d9850b40d793bd2503504332b9c18118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:09:05.720 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:05.319411904 +0000 UTC m=+0.421653025 container init 4f4bed63695a7ff3dd11f6b8a0fa67a9d9850b40d793bd2503504332b9c18118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:09:05.720 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:05.322867855 +0000 UTC m=+0.425108981 container start 4f4bed63695a7ff3dd11f6b8a0fa67a9d9850b40d793bd2503504332b9c18118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:09:05.721 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:05.452935316 +0000 UTC m=+0.555176456 container attach 4f4bed63695a7ff3dd11f6b8a0fa67a9d9850b40d793bd2503504332b9c18118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:09:06.220 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 12 03:09:05 smithi196 podman[72131]: 2024-09-12 03:09:05.863550489 +0000 UTC m=+0.965791611 container died 4f4bed63695a7ff3dd11f6b8a0fa67a9d9850b40d793bd2503504332b9c18118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:09:06.589 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:09:06.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:06.630 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:09:06.630 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:09:06.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:09:06.954 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:06 smithi046 systemd[1]: Stopping Ceph osd.3 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:07.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T03:09:06.954+0000 7f1286db7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:07.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T03:09:06.954+0000 7f1286db7640 -1 osd.3 122 *** Got signal Terminated *** 2024-09-12T03:09:07.283 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:06 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3[51815]: 2024-09-12T03:09:06.954+0000 7f1286db7640 -1 osd.3 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:13.031 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:12 smithi046 podman[109717]: 2024-09-12 03:09:12.725428644 +0000 UTC m=+5.895386815 container died f0905a7c01460317d03a21be0513643916b9e9040767938f1e639b3beb54ec34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:09:13.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109717]: 2024-09-12 03:09:13.121293986 +0000 UTC m=+6.291252156 container cleanup f0905a7c01460317d03a21be0513643916b9e9040767938f1e639b3beb54ec34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:09:13.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 bash[109717]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:09:13.731 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109732]: 2024-09-12 03:09:13.414728474 +0000 UTC m=+0.686106055 container remove f0905a7c01460317d03a21be0513643916b9e9040767938f1e639b3beb54ec34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:09:13.993 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109819]: 2024-09-12 03:09:13.634790935 +0000 UTC m=+0.016421706 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:13.993 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109819]: 2024-09-12 03:09:13.732061466 +0000 UTC m=+0.113692236 container create 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:09:14.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109819]: 2024-09-12 03:09:13.993106933 +0000 UTC m=+0.374737704 container init 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:09:14.282 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:13 smithi046 podman[109819]: 2024-09-12 03:09:13.997340947 +0000 UTC m=+0.378971722 container start 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:09:14.283 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 podman[109819]: 2024-09-12 03:09:14.068133575 +0000 UTC m=+0.449764356 container attach 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T03:09:14.283 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 podman[109819]: 2024-09-12 03:09:14.193159661 +0000 UTC m=+0.574790432 container died 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:09:14.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:09:14.781 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 podman[109842]: 2024-09-12 03:09:14.738904064 +0000 UTC m=+0.539488434 container remove 7e3805601f31027f97f6a192294c310d64caa000254e78a5449d68de511005bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:09:14.781 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-12T03:09:14.781 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 systemd[1]: Stopped Ceph osd.3 for 2691fc0c-70b0-11ef-bcea-c7b262605968. 2024-09-12T03:09:14.782 INFO:journalctl@ceph.osd.3.smithi046.stdout:Sep 12 03:09:14 smithi046 systemd[1]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.3.service: Consumed 10min 31.823s CPU time. 2024-09-12T03:09:14.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:14.798 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:09:14.798 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:09:14.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:09:15.110 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:14 smithi145 systemd[1]: Stopping Ceph osd.4 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:15.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:15 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T03:09:15.111+0000 7f0b67af3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:15.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:15 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T03:09:15.111+0000 7f0b67af3640 -1 osd.4 122 *** Got signal Terminated *** 2024-09-12T03:09:15.610 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:15 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4[46007]: 2024-09-12T03:09:15.111+0000 7f0b67af3640 -1 osd.4 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:21.088 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:20 smithi145 podman[71794]: 2024-09-12 03:09:20.775000572 +0000 UTC m=+5.781885035 container died 973823a80914cde9c0532fcdf75e9ca8e415bb98cfe0abca7dd572451060b2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:09:21.360 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:21 smithi145 podman[71794]: 2024-09-12 03:09:21.20392336 +0000 UTC m=+6.210807819 container cleanup 973823a80914cde9c0532fcdf75e9ca8e415bb98cfe0abca7dd572451060b2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:21.360 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:21 smithi145 bash[71794]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:09:21.860 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:21 smithi145 podman[71809]: 2024-09-12 03:09:21.480603652 +0000 UTC m=+0.696335018 container remove 973823a80914cde9c0532fcdf75e9ca8e415bb98cfe0abca7dd572451060b2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:09:22.259 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:21 smithi145 podman[71895]: 2024-09-12 03:09:21.865621717 +0000 UTC m=+0.015997168 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:22.259 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:21 smithi145 podman[71895]: 2024-09-12 03:09:21.981675729 +0000 UTC m=+0.132051168 container create d3d2ca20e31419e999e8ee867b6960414283e9734c80b7f0b39ee764e2f4d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:09:22.511 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:22 smithi145 podman[71895]: 2024-09-12 03:09:22.259343794 +0000 UTC m=+0.409719234 container init d3d2ca20e31419e999e8ee867b6960414283e9734c80b7f0b39ee764e2f4d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:09:22.511 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:22 smithi145 podman[71895]: 2024-09-12 03:09:22.263518846 +0000 UTC m=+0.413894285 container start d3d2ca20e31419e999e8ee867b6960414283e9734c80b7f0b39ee764e2f4d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:09:22.511 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:22 smithi145 podman[71895]: 2024-09-12 03:09:22.367850469 +0000 UTC m=+0.518225922 container attach d3d2ca20e31419e999e8ee867b6960414283e9734c80b7f0b39ee764e2f4d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:09:22.512 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 12 03:09:22 smithi145 podman[71895]: 2024-09-12 03:09:22.472146698 +0000 UTC m=+0.622522149 container died d3d2ca20e31419e999e8ee867b6960414283e9734c80b7f0b39ee764e2f4d339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:09:23.130 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:09:23.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:23.169 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:09:23.169 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:09:23.170 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:09:23.470 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:23 smithi196 systemd[1]: Stopping Ceph osd.5 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:23.970 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:23 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T03:09:23.508+0000 7fd1d825d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:23.970 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:23 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T03:09:23.508+0000 7fd1d825d640 -1 osd.5 122 *** Got signal Terminated *** 2024-09-12T03:09:23.970 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:23 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5[46153]: 2024-09-12T03:09:23.508+0000 7fd1d825d640 -1 osd.5 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:29.638 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:29 smithi196 podman[72235]: 2024-09-12 03:09:29.360787915 +0000 UTC m=+5.986344221 container died fef5b2a50e7b246efe22a30af62cb858e4fe4696c736849445dcf9c83b92d2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:09:29.970 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:29 smithi196 podman[72235]: 2024-09-12 03:09:29.784378675 +0000 UTC m=+6.409934989 container cleanup fef5b2a50e7b246efe22a30af62cb858e4fe4696c736849445dcf9c83b92d2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:29.970 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:29 smithi196 bash[72235]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:09:30.400 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72253]: 2024-09-12 03:09:30.061237234 +0000 UTC m=+0.688882400 container remove fef5b2a50e7b246efe22a30af62cb858e4fe4696c736849445dcf9c83b92d2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:30.651 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72351]: 2024-09-12 03:09:30.303001037 +0000 UTC m=+0.016380367 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:30.651 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72351]: 2024-09-12 03:09:30.403640748 +0000 UTC m=+0.117020069 container create 7898df19c182f65239aff07f2627553e028c19f22b52db67e2ebc8b6badfd90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:09:30.905 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72351]: 2024-09-12 03:09:30.68980369 +0000 UTC m=+0.403183010 container init 7898df19c182f65239aff07f2627553e028c19f22b52db67e2ebc8b6badfd90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:09:30.905 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72351]: 2024-09-12 03:09:30.693630129 +0000 UTC m=+0.407009453 container start 7898df19c182f65239aff07f2627553e028c19f22b52db67e2ebc8b6badfd90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:09:30.905 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72351]: 2024-09-12 03:09:30.789766937 +0000 UTC m=+0.503146261 container attach 7898df19c182f65239aff07f2627553e028c19f22b52db67e2ebc8b6badfd90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:30.905 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 12 03:09:30 smithi196 podman[72376]: 2024-09-12 03:09:30.90532616 +0000 UTC m=+0.017059515 container died 7898df19c182f65239aff07f2627553e028c19f22b52db67e2ebc8b6badfd90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-12T03:09:31.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:09:31.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:31.618 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:09:31.618 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:09:31.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:09:31.933 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:31 smithi046 systemd[1]: Stopping Ceph osd.6 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:32.282 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T03:09:31.932+0000 7f5cd01f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:32.282 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T03:09:31.932+0000 7f5cd01f0640 -1 osd.6 122 *** Got signal Terminated *** 2024-09-12T03:09:32.282 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:31 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6[58695]: 2024-09-12T03:09:31.932+0000 7f5cd01f0640 -1 osd.6 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:37.782 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:37 smithi046 podman[109922]: 2024-09-12 03:09:37.495592405 +0000 UTC m=+5.678722818 container died 834a875ed1746c052ebdced9c1f249299a55fd0437a25181bf99c24058146191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:38.067 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:37 smithi046 podman[109922]: 2024-09-12 03:09:37.840924176 +0000 UTC m=+6.024054579 container cleanup 834a875ed1746c052ebdced9c1f249299a55fd0437a25181bf99c24058146191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:38.067 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:37 smithi046 bash[109922]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:09:38.489 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:38 smithi046 podman[109951]: 2024-09-12 03:09:38.067475932 +0000 UTC m=+0.565519527 container remove 834a875ed1746c052ebdced9c1f249299a55fd0437a25181bf99c24058146191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:09:38.775 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:38 smithi046 podman[110039]: 2024-09-12 03:09:38.392231978 +0000 UTC m=+0.016531487 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:38.775 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:38 smithi046 podman[110039]: 2024-09-12 03:09:38.647867779 +0000 UTC m=+0.272167319 container create 4bd2ba5d6feb54f78a765220e5553c9631adb35e4da59b2974e42ff3501c0cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:39.032 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:38 smithi046 podman[110039]: 2024-09-12 03:09:38.913172605 +0000 UTC m=+0.537472103 container init 4bd2ba5d6feb54f78a765220e5553c9631adb35e4da59b2974e42ff3501c0cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:09:39.032 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:38 smithi046 podman[110039]: 2024-09-12 03:09:38.916577658 +0000 UTC m=+0.540877154 container start 4bd2ba5d6feb54f78a765220e5553c9631adb35e4da59b2974e42ff3501c0cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:09:39.032 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:39 smithi046 podman[110039]: 2024-09-12 03:09:39.004785427 +0000 UTC m=+0.629084935 container attach 4bd2ba5d6feb54f78a765220e5553c9631adb35e4da59b2974e42ff3501c0cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:09:39.431 INFO:journalctl@ceph.osd.6.smithi046.stdout:Sep 12 03:09:39 smithi046 podman[110039]: 2024-09-12 03:09:39.113249974 +0000 UTC m=+0.737549480 container died 4bd2ba5d6feb54f78a765220e5553c9631adb35e4da59b2974e42ff3501c0cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:09:39.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:09:39.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:39.778 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:09:39.778 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:09:39.778 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:09:40.099 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:39 smithi145 systemd[1]: Stopping Ceph osd.7 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:40.360 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T03:09:40.098+0000 7fa70d436640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:40.360 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T03:09:40.098+0000 7fa70d436640 -1 osd.7 122 *** Got signal Terminated *** 2024-09-12T03:09:40.360 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:40 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7[52627]: 2024-09-12T03:09:40.098+0000 7fa70d436640 -1 osd.7 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:46.112 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:45 smithi145 podman[72010]: 2024-09-12 03:09:45.85169465 +0000 UTC m=+5.870610479 container died e3b76555152bdbf476ea3f6a11d0273938830053ad4649d19d38ffcf72979a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:46.524 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:46 smithi145 podman[72010]: 2024-09-12 03:09:46.26533818 +0000 UTC m=+6.284254009 container cleanup e3b76555152bdbf476ea3f6a11d0273938830053ad4649d19d38ffcf72979a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:09:46.525 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:46 smithi145 bash[72010]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:09:46.860 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:46 smithi145 podman[72025]: 2024-09-12 03:09:46.524960691 +0000 UTC m=+0.668017847 container remove e3b76555152bdbf476ea3f6a11d0273938830053ad4649d19d38ffcf72979a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:46.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:46 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:46.525+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:46.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:46 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:46.585+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:47.195 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:46 smithi145 podman[72116]: 2024-09-12 03:09:46.836808455 +0000 UTC m=+0.016250176 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:47.195 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:46 smithi145 podman[72116]: 2024-09-12 03:09:46.967530315 +0000 UTC m=+0.146972038 container create af0943aa0e76dcf02732dbf92c4e15126c5fc530e5d4d7ad80fb31cb1182c547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:09:47.195 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:47 smithi145 podman[72116]: 2024-09-12 03:09:47.194996939 +0000 UTC m=+0.374438661 container init af0943aa0e76dcf02732dbf92c4e15126c5fc530e5d4d7ad80fb31cb1182c547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:09:47.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:47 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:47.151+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:47.551 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:47 smithi145 podman[72116]: 2024-09-12 03:09:47.198447671 +0000 UTC m=+0.377889398 container start af0943aa0e76dcf02732dbf92c4e15126c5fc530e5d4d7ad80fb31cb1182c547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:09:47.551 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:47 smithi145 podman[72116]: 2024-09-12 03:09:47.345394361 +0000 UTC m=+0.524836100 container attach af0943aa0e76dcf02732dbf92c4e15126c5fc530e5d4d7ad80fb31cb1182c547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:09:47.551 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 12 03:09:47 smithi145 podman[72116]: 2024-09-12 03:09:47.383174993 +0000 UTC m=+0.562616724 container died af0943aa0e76dcf02732dbf92c4e15126c5fc530e5d4d7ad80fb31cb1182c547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:09:47.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:47 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:47.551+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:47.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:47 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:47.553+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:48.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:09:48.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:48.048 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:09:48.048 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:09:48.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:09:48.362 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:48 smithi196 systemd[1]: Stopping Ceph osd.8 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:48.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:48 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:48.104+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:48.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:48 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:48.603+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:48.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:48 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T03:09:48.361+0000 7fd703ee3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:48.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:48 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T03:09:48.361+0000 7fd703ee3640 -1 osd.8 122 *** Got signal Terminated *** 2024-09-12T03:09:48.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:48 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8[52643]: 2024-09-12T03:09:48.361+0000 7fd703ee3640 -1 osd.8 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:48.864 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:48 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:48.534+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:49.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:49 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:49.103+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:49.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:49 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:49.541+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:49.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:49 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:49.597+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:50.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:50 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:50.102+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:50.859 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:50 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:50.588+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:50.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:50 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:50.560+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:51.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:51 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:51.083+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:51.762 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:51 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:51.608+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:51.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:51 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:51.511+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:52.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:52 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:52.101+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:52.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:52 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:52.600+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:52.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:52 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:52.536+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:53.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:53 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:53.079+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:53.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:53 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:53.582+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:53.970 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:53 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:53.492+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:54.469 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:54 smithi196 podman[72454]: 2024-09-12 03:09:54.165396027 +0000 UTC m=+5.924429741 container died 85f9ebd08e75daa4c7b9adc878362d8ebb604cc9097040c7e3197c9e29e7c634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T03:09:54.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:54 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:54.041+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:54.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:54 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:54.469+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:54.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:54 smithi196 podman[72454]: 2024-09-12 03:09:54.478165306 +0000 UTC m=+6.237199067 container cleanup 85f9ebd08e75daa4c7b9adc878362d8ebb604cc9097040c7e3197c9e29e7c634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:54.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:54 smithi196 bash[72454]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:09:54.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:54 smithi196 podman[72469]: 2024-09-12 03:09:54.712594521 +0000 UTC m=+0.538032627 container remove 85f9ebd08e75daa4c7b9adc878362d8ebb604cc9097040c7e3197c9e29e7c634 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T03:09:54.859 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:54 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:54.605+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:55.299 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:55.029936089 +0000 UTC m=+0.112717557 container create 74bb7f1bb5528b31775d554a5c40719db6a14bd3added9514ba24ca338279e68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:55.299 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:54.933290576 +0000 UTC m=+0.016072059 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:09:55.299 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:55.299767873 +0000 UTC m=+0.382549350 container init 74bb7f1bb5528b31775d554a5c40719db6a14bd3added9514ba24ca338279e68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:09:55.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:55 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:55.086+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:55.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:55 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:55.422+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:55.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:55.303408621 +0000 UTC m=+0.386190098 container start 74bb7f1bb5528b31775d554a5c40719db6a14bd3added9514ba24ca338279e68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T03:09:55.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:55.391132863 +0000 UTC m=+0.473914345 container attach 74bb7f1bb5528b31775d554a5c40719db6a14bd3added9514ba24ca338279e68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:09:55.720 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 12 03:09:55 smithi196 podman[72557]: 2024-09-12 03:09:55.495111721 +0000 UTC m=+0.577893208 container died 74bb7f1bb5528b31775d554a5c40719db6a14bd3added9514ba24ca338279e68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:09:55.859 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:55 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:55.561+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:56.152 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:09:56.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:09:56.193 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:09:56.193 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:09:56.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:09:56.519 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:56.039+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:56.519 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:56 smithi046 systemd[1]: Stopping Ceph osd.9 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:09:56.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:56 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:56.410+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:56.782 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:56.518+0000 7f02bcfa5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:56.782 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:56.518+0000 7f02bcfa5640 -1 osd.9 122 *** Got signal Terminated *** 2024-09-12T03:09:56.782 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:56 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:56.518+0000 7f02bcfa5640 -1 osd.9 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:09:56.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:56 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:56.553+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:57.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:57 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:57.078+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:57.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:57 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:57.437+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:57.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:57 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:57.537+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:57.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:57 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:57.537+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:09:58.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:58 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:58.127+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:58.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:58.387+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:58.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:58 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:58.529+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:58.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:58 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:58.529+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:09:59.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:09:59 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:09:59.128+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:09:59.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:09:59 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:09:59.364+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:09:59.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:59 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:59.564+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:09:59.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:09:59 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:09:59.564+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:00.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:10:00.110+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:10:00.532 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:00 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:10:00.110+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:53.272831+0000 front 2024-09-12T03:08:53.272817+0000 (oldest deadline 2024-09-12T03:09:59.172592+0000) 2024-09-12T03:10:00.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:00 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:00.355+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:00.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:00 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:00.542+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:00.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:00 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:00.542+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:01.433 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:01 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:10:01.085+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:39.970239+0000 front 2024-09-12T03:08:39.970278+0000 (oldest deadline 2024-09-12T03:09:46.970766+0000) 2024-09-12T03:10:01.434 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:01 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9[65210]: 2024-09-12T03:10:01.085+0000 7f02b95bb640 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:53.272831+0000 front 2024-09-12T03:08:53.272817+0000 (oldest deadline 2024-09-12T03:09:59.172592+0000) 2024-09-12T03:10:01.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:01 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:01.390+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:01.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:01 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:01.527+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:01.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:01 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:01.527+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:02.720 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:02 smithi046 podman[110140]: 2024-09-12 03:10:02.443487333 +0000 UTC m=+6.041246913 container died 4a4f4f243e961df21d4e8f5cc9a5de1b5b70374b5b7975522ab99a9f894df26c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T03:10:02.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:02 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:02.372+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:02.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:02 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:02.517+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:02.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:02 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:02.517+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:03.032 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:02 smithi046 podman[110140]: 2024-09-12 03:10:02.79440559 +0000 UTC m=+6.392165168 container cleanup 4a4f4f243e961df21d4e8f5cc9a5de1b5b70374b5b7975522ab99a9f894df26c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:10:03.032 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:02 smithi046 bash[110140]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:10:03.416 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110156]: 2024-09-12 03:10:03.079205697 +0000 UTC m=+0.626729330 container remove 4a4f4f243e961df21d4e8f5cc9a5de1b5b70374b5b7975522ab99a9f894df26c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:10:03.673 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.31938852 +0000 UTC m=+0.016800648 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:10:03.674 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.421488266 +0000 UTC m=+0.118900392 container create a64f0ba3099d8bee5da2280bca5e6f62279a7f92dd6e5e8479cd38fbc8f4ca03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:10:03.674 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.67426317 +0000 UTC m=+0.371675296 container init a64f0ba3099d8bee5da2280bca5e6f62279a7f92dd6e5e8479cd38fbc8f4ca03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:10:03.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:03 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:03.339+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:03.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:03 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:03.559+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:03.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:03 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:03.559+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:04.032 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.67773132 +0000 UTC m=+0.375143452 container start a64f0ba3099d8bee5da2280bca5e6f62279a7f92dd6e5e8479cd38fbc8f4ca03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T03:10:04.033 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.757590624 +0000 UTC m=+0.455002752 container attach a64f0ba3099d8bee5da2280bca5e6f62279a7f92dd6e5e8479cd38fbc8f4ca03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T03:10:04.033 INFO:journalctl@ceph.osd.9.smithi046.stdout:Sep 12 03:10:03 smithi046 podman[110244]: 2024-09-12 03:10:03.867347736 +0000 UTC m=+0.564759864 container died a64f0ba3099d8bee5da2280bca5e6f62279a7f92dd6e5e8479cd38fbc8f4ca03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:04.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:10:04.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:04.538 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:10:04.538 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:10:04.539 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:10:04.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:04 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:04.328+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:04.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 systemd[1]: Stopping Ceph osd.10 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:04.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:04.607+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:04.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:04.607+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:05.360 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:04.866+0000 7f5c701e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:05.360 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:04.866+0000 7f5c701e4640 -1 osd.10 122 *** Got signal Terminated *** 2024-09-12T03:10:05.360 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:04 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:04.866+0000 7f5c701e4640 -1 osd.10 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:10:05.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:05 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:05.287+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:06.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:05.637+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:06.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:05 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:05.637+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:06.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:06.280+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:06.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:06.280+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:07.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:06 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:06.625+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:07.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:06 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:06.625+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:07.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:07 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:07.283+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:07.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:07 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:07.283+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:07.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:07 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:07.597+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:07.860 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:07 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:07.597+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:08.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:08.294+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:08.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:08 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:08.294+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:09.109 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:08.625+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:09.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:08.625+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:09.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:08 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:08.625+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.699592+0000 front 2024-09-12T03:09:02.699496+0000 (oldest deadline 2024-09-12T03:10:08.599635+0000) 2024-09-12T03:10:09.674 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:09.270+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:09.674 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:09 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:09.270+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:10.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:09.638+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:42.896081+0000 front 2024-09-12T03:08:42.895982+0000 (oldest deadline 2024-09-12T03:09:45.796039+0000) 2024-09-12T03:10:10.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:09.638+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.145:6804 osd.1 since back 2024-09-12T03:08:49.297173+0000 front 2024-09-12T03:08:49.297199+0000 (oldest deadline 2024-09-12T03:09:57.497976+0000) 2024-09-12T03:10:10.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:09 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10[59141]: 2024-09-12T03:10:09.638+0000 7f5c687f2640 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.699592+0000 front 2024-09-12T03:09:02.699496+0000 (oldest deadline 2024-09-12T03:10:08.599635+0000) 2024-09-12T03:10:10.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:10 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:10.272+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:10.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:10 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:10.272+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:10.768 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:10 smithi145 podman[72233]: 2024-09-12 03:10:10.445754213 +0000 UTC m=+5.707035281 container died 02b330a2c4e5905f22f7d583827c9b2fe7d9dde5413afa058be7fa3d8c8a3b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:10:11.101 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:10 smithi145 podman[72233]: 2024-09-12 03:10:10.866886068 +0000 UTC m=+6.128167123 container cleanup 02b330a2c4e5905f22f7d583827c9b2fe7d9dde5413afa058be7fa3d8c8a3b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:11.101 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:10 smithi145 bash[72233]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:10:11.360 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72250]: 2024-09-12 03:10:11.101436151 +0000 UTC m=+0.641100602 container remove 02b330a2c4e5905f22f7d583827c9b2fe7d9dde5413afa058be7fa3d8c8a3b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:10:11.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:11 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:11.260+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:11.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:11 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:11.260+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:11.846 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.430610604 +0000 UTC m=+0.016253971 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:10:11.846 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.544245448 +0000 UTC m=+0.129888851 container create b311c2d8bca5f4c10013d8e2d72a238d26457b2fe3eaf2d48d78e6fef577239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:10:11.846 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.763767606 +0000 UTC m=+0.349410967 container init b311c2d8bca5f4c10013d8e2d72a238d26457b2fe3eaf2d48d78e6fef577239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:10:11.846 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.767061485 +0000 UTC m=+0.352704854 container start b311c2d8bca5f4c10013d8e2d72a238d26457b2fe3eaf2d48d78e6fef577239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:10:11.846 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.846339656 +0000 UTC m=+0.431983033 container attach b311c2d8bca5f4c10013d8e2d72a238d26457b2fe3eaf2d48d78e6fef577239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:10:12.110 INFO:journalctl@ceph.osd.10.smithi145.stdout:Sep 12 03:10:11 smithi145 podman[72337]: 2024-09-12 03:10:11.951023321 +0000 UTC m=+0.536666688 container died b311c2d8bca5f4c10013d8e2d72a238d26457b2fe3eaf2d48d78e6fef577239f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:10:12.542 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:10:12.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:12.582 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:10:12.582 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:10:12.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:10:12.612 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:12.277+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:12.612 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:12.277+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:12.896 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 systemd[1]: Stopping Ceph osd.11 for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:13.220 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:12.895+0000 7f9905521640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:13.220 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:12.895+0000 7f9905521640 -1 osd.11 122 *** Got signal Terminated *** 2024-09-12T03:10:13.220 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:12 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:12.895+0000 7f9905521640 -1 osd.11 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:10:13.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:13 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:13.280+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:13.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:13 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:13.280+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:14.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:14 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:14.241+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:14.720 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:14 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:14.241+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:15.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:15 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:15.204+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:15.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:15 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:15.204+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:16.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:16 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:16.170+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:16.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:16 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:16.170+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:16.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:16 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:16.170+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-12T03:09:07.317657+0000 front 2024-09-12T03:09:07.317487+0000 (oldest deadline 2024-09-12T03:10:15.418678+0000) 2024-09-12T03:10:17.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:17 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:17.217+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-12T03:08:41.713380+0000 front 2024-09-12T03:08:41.713473+0000 (oldest deadline 2024-09-12T03:09:45.813576+0000) 2024-09-12T03:10:17.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:17 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:17.217+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-12T03:09:02.116326+0000 front 2024-09-12T03:09:02.116445+0000 (oldest deadline 2024-09-12T03:10:06.216446+0000) 2024-09-12T03:10:17.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:17 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11[59599]: 2024-09-12T03:10:17.217+0000 7f9901b37640 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-09-12T03:09:07.317657+0000 front 2024-09-12T03:09:07.317487+0000 (oldest deadline 2024-09-12T03:10:15.418678+0000) 2024-09-12T03:10:19.114 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:18 smithi196 podman[72672]: 2024-09-12 03:10:18.843819912 +0000 UTC m=+6.052685327 container died 622161dde0d9184f40e5d3dded0b2e062210472c8b641747da15583b9b46c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:10:19.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:19 smithi196 podman[72672]: 2024-09-12 03:10:19.220943446 +0000 UTC m=+6.429808861 container cleanup 622161dde0d9184f40e5d3dded0b2e062210472c8b641747da15583b9b46c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-12T03:10:19.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:19 smithi196 bash[72672]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:10:19.805 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:19 smithi196 podman[72689]: 2024-09-12 03:10:19.497966498 +0000 UTC m=+0.648637530 container remove 622161dde0d9184f40e5d3dded0b2e062210472c8b641747da15583b9b46c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:20.100 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:19 smithi196 podman[72780]: 2024-09-12 03:10:19.708468732 +0000 UTC m=+0.015789577 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:10:20.100 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:19 smithi196 podman[72780]: 2024-09-12 03:10:19.80602518 +0000 UTC m=+0.113346018 container create cf61c194fd4a0bba32fbbffca1b285363acf6569d90a4ba5a1dc2abd66a23905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T03:10:20.100 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:20 smithi196 podman[72780]: 2024-09-12 03:10:20.033351637 +0000 UTC m=+0.340672476 container init cf61c194fd4a0bba32fbbffca1b285363acf6569d90a4ba5a1dc2abd66a23905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:10:20.100 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:20 smithi196 podman[72780]: 2024-09-12 03:10:20.036615422 +0000 UTC m=+0.343936271 container start cf61c194fd4a0bba32fbbffca1b285363acf6569d90a4ba5a1dc2abd66a23905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:10:20.100 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:20 smithi196 podman[72780]: 2024-09-12 03:10:20.099939172 +0000 UTC m=+0.407260032 container attach cf61c194fd4a0bba32fbbffca1b285363acf6569d90a4ba5a1dc2abd66a23905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:10:20.470 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 12 03:10:20 smithi196 podman[72780]: 2024-09-12 03:10:20.226551387 +0000 UTC m=+0.533872234 container died cf61c194fd4a0bba32fbbffca1b285363acf6569d90a4ba5a1dc2abd66a23905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:10:20.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:10:20.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:20.908 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:10:20.909 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:10:20.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:10:21.212 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:20 smithi046 systemd[1]: Stopping Ceph mds.a for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:21.532 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:21 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a[72102]: 2024-09-12T03:10:21.212+0000 7fc237a74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:21.532 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:21 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a[72102]: 2024-09-12T03:10:21.212+0000 7fc237a74640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:10:24.663 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:24 smithi046 podman[110345]: 2024-09-12 03:10:24.353939258 +0000 UTC m=+3.245580205 container died 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:10:25.030 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:24 smithi046 podman[110345]: 2024-09-12 03:10:24.753903766 +0000 UTC m=+3.645544701 container cleanup 9b2e540d37f862f3561a44b7be96c2b7fbe679158df92d1889e293e665b78b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:10:25.030 INFO:journalctl@ceph.mds.a.smithi046.stdout:Sep 12 03:10:24 smithi046 bash[110345]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:10:25.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:10:25.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:25.203 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:10:25.204 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:10:25.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:10:25.782 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:25 smithi046 systemd[1]: Stopping Ceph mds.d for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:25.782 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:25 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d[70879]: 2024-09-12T03:10:25.524+0000 7f77de3a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:25.782 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:25 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d[70879]: 2024-09-12T03:10:25.524+0000 7f77de3a8640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:10:28.782 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:28 smithi046 podman[110466]: 2024-09-12 03:10:28.511751451 +0000 UTC m=+3.085023710 container died 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:10:29.242 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:29 smithi046 podman[110466]: 2024-09-12 03:10:28.999968163 +0000 UTC m=+3.573240408 container cleanup 334f22ba3c596cc39a5b6a184eaf367e0aaa5c46b62454dbaea230d555cd17d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:10:29.242 INFO:journalctl@ceph.mds.d.smithi046.stdout:Sep 12 03:10:29 smithi046 bash[110466]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:10:29.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:10:29.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:29.416 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:10:29.416 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:10:29.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:10:29.782 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 12 03:10:29 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-g[71762]: 2024-09-12T03:10:29.753+0000 7f6ef19ec640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:10:33.387 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 12 03:10:33 smithi046 podman[110586]: 2024-09-12 03:10:33.110849743 +0000 UTC m=+3.463025519 container died 0f0a7aecd6560106d19483b26a5254cb56a5c7914d4c6e56373d6cc51950882e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:10:33.782 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 12 03:10:33 smithi046 podman[110586]: 2024-09-12 03:10:33.478412339 +0000 UTC m=+3.830588118 container cleanup 0f0a7aecd6560106d19483b26a5254cb56a5c7914d4c6e56373d6cc51950882e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:10:33.782 INFO:journalctl@ceph.mds.g.smithi046.stdout:Sep 12 03:10:33 smithi046 bash[110586]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:10:34.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:10:34.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:34.097 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:10:34.097 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:10:34.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:10:34.424 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:34 smithi046 systemd[1]: Stopping Ceph mds.j for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:34.782 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j[70483]: 2024-09-12T03:10:34.424+0000 7fa234135640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:34.782 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:34 smithi046 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j[70483]: 2024-09-12T03:10:34.424+0000 7fa234135640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:10:37.280 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:36 smithi046 podman[110708]: 2024-09-12 03:10:36.89486121 +0000 UTC m=+2.578918739 container died 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:37.532 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:37 smithi046 podman[110708]: 2024-09-12 03:10:37.382650978 +0000 UTC m=+3.066708509 container cleanup 10e27fc94aa44d66301b80422409336341a3c50f606e408992c2415eb5936130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T03:10:37.532 INFO:journalctl@ceph.mds.j.smithi046.stdout:Sep 12 03:10:37 smithi046 bash[110708]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:10:37.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:10:37.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:37.816 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:10:37.816 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:10:37.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:10:38.101 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:37 smithi145 systemd[1]: Stopping Ceph mds.b for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:38.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:38 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: 2024-09-12T03:10:38.135+0000 7f9774698640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:38.360 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:38 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b[64313]: 2024-09-12T03:10:38.135+0000 7f9774698640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:10:41.484 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:41 smithi145 podman[72441]: 2024-09-12 03:10:41.201246837 +0000 UTC m=+3.192733392 container died 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:10:41.860 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:41 smithi145 podman[72441]: 2024-09-12 03:10:41.566276062 +0000 UTC m=+3.557762616 container cleanup 6e329f6a256e65c89b6d3a81188ddd8383c428b7ee7601ad3db0c9f3e64b0ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:10:41.860 INFO:journalctl@ceph.mds.b.smithi145.stdout:Sep 12 03:10:41 smithi145 bash[72441]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:10:42.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:10:42.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:42.045 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:10:42.045 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:10:42.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:10:42.386 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:42 smithi145 systemd[1]: Stopping Ceph mds.e for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:42.859 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:42 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e[63706]: 2024-09-12T03:10:42.386+0000 7f9a96712640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:42.860 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:42 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e[63706]: 2024-09-12T03:10:42.386+0000 7f9a96712640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:10:46.705 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:46 smithi145 podman[72576]: 2024-09-12 03:10:46.400589942 +0000 UTC m=+4.135462102 container died 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:47.029 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:46 smithi145 podman[72576]: 2024-09-12 03:10:46.786383495 +0000 UTC m=+4.521255655 container cleanup 8a109ebc0bf35798725775274ed60396b970ecd7c2e3bfdd305629c492a90570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:10:47.029 INFO:journalctl@ceph.mds.e.smithi145.stdout:Sep 12 03:10:46 smithi145 bash[72576]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:10:47.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:10:47.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:47.199 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:10:47.199 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:10:47.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:10:47.359 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:47 smithi145 systemd[1]: Stopping Ceph mds.h for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:47.860 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:47 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: 2024-09-12T03:10:47.514+0000 7f02935d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:47.860 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:47 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h[64007]: 2024-09-12T03:10:47.514+0000 7f02935d0640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:10:49.474 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:49 smithi145 podman[72698]: 2024-09-12 03:10:49.194490261 +0000 UTC m=+1.780055162 container died 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-12T03:10:49.825 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:49 smithi145 podman[72698]: 2024-09-12 03:10:49.565551898 +0000 UTC m=+2.151116798 container cleanup 2fd53589e0976e0324e155e6c6199cb442b68ca9d220568e332e4fd7ab704390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:10:49.825 INFO:journalctl@ceph.mds.h.smithi145.stdout:Sep 12 03:10:49 smithi145 bash[72698]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:10:49.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:10:50.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:50.011 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:10:50.011 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:10:50.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:10:50.359 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:50 smithi145 systemd[1]: Stopping Ceph mds.k for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:50.859 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:50 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k[63395]: 2024-09-12T03:10:50.369+0000 7f9fb2a52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:50.860 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:50 smithi145 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k[63395]: 2024-09-12T03:10:50.369+0000 7f9fb2a52640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:10:53.894 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:53 smithi145 podman[72818]: 2024-09-12 03:10:53.628064102 +0000 UTC m=+3.405459339 container died 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:10:54.169 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:53 smithi145 podman[72818]: 2024-09-12 03:10:53.968963343 +0000 UTC m=+3.746358584 container cleanup 6aec3e4d8a6b4219e8800073c2cce8d1033cc8750ce52d02995fb9d7535fb10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:10:54.170 INFO:journalctl@ceph.mds.k.smithi145.stdout:Sep 12 03:10:53 smithi145 bash[72818]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:10:54.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:10:54.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:54.423 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:10:54.423 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:10:54.423 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:10:54.720 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:54 smithi196 systemd[1]: Stopping Ceph mds.c for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:55.220 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:54 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c[64394]: 2024-09-12T03:10:54.725+0000 7f3bd7a6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:55.220 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:54 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c[64394]: 2024-09-12T03:10:54.725+0000 7f3bd7a6b640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:10:57.452 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:57 smithi196 podman[72882]: 2024-09-12 03:10:57.192923256 +0000 UTC m=+2.574054696 container died 1626c637f9f7183d30d016dbfd97c0e84ca1d7c39cdd9fb6012490e4a16b9452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:10:57.720 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:57 smithi196 podman[72882]: 2024-09-12 03:10:57.551183468 +0000 UTC m=+2.932314913 container cleanup 1626c637f9f7183d30d016dbfd97c0e84ca1d7c39cdd9fb6012490e4a16b9452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-12T03:10:57.720 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 12 03:10:57 smithi196 bash[72882]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:10:57.985 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:10:58.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:10:58.021 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:10:58.022 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:10:58.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:10:58.470 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:10:58 smithi196 systemd[1]: Stopping Ceph mds.f for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:10:58.470 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:10:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: 2024-09-12T03:10:58.346+0000 7f00cafde640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:10:58.470 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:10:58 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f[64090]: 2024-09-12T03:10:58.346+0000 7f00cafde640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:11:01.470 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:11:01 smithi196 podman[73016]: 2024-09-12 03:11:01.186704625 +0000 UTC m=+2.944617619 container died 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:11:01.830 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:11:01 smithi196 podman[73016]: 2024-09-12 03:11:01.563020725 +0000 UTC m=+3.320933718 container cleanup 6d931c7b2974357c8156432fc81f879bf90f821f578e34f4d9c11b4682abf9e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:11:01.831 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 12 03:11:01 smithi196 bash[73016]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:11:01.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:11:02.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:11:02.027 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:11:02.027 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:11:02.027 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:11:02.350 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:02 smithi196 systemd[1]: Stopping Ceph mds.i for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:11:02.720 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:02 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: 2024-09-12T03:11:02.349+0000 7f8e2ca97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:11:02.720 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:02 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i[64707]: 2024-09-12T03:11:02.349+0000 7f8e2ca97640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:11:05.467 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:05 smithi196 podman[73137]: 2024-09-12 03:11:05.189226187 +0000 UTC m=+2.945335048 container died d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:11:05.720 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:05 smithi196 podman[73137]: 2024-09-12 03:11:05.549690113 +0000 UTC m=+3.305798973 container cleanup d3d11b84434f8cd554a0066b770ede8ab8a13f2b8eefd3baecaf07e1ba69da1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:11:05.720 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 12 03:11:05 smithi196 bash[73137]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:11:05.960 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:11:05.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:11:05.993 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:11:05.993 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:11:05.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:11:06.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:06 smithi196 systemd[1]: Stopping Ceph mds.l for 2691fc0c-70b0-11ef-bcea-c7b262605968... 2024-09-12T03:11:06.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: 2024-09-12T03:11:06.293+0000 7f38c0862640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:11:06.470 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:06 smithi196 ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l[63789]: 2024-09-12T03:11:06.293+0000 7f38c0862640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:11:09.454 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:09 smithi196 podman[73254]: 2024-09-12 03:11:09.185786569 +0000 UTC m=+2.968155549 container died 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:11:09.720 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:09 smithi196 podman[73254]: 2024-09-12 03:11:09.561244137 +0000 UTC m=+3.343613113 container cleanup 237f921b7c9cc18e1df80de0308295884f47c9a488f83ff3959ecaeb0aa8710c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:11:09.720 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 12 03:11:09 smithi196 bash[73254]: ceph-2691fc0c-70b0-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:11:09.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2691fc0c-70b0-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:11:10.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:11:10.016 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:11:10.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:11:10.294 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:11:12.743 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:11:12.944 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:11:15.293 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:11:15.492 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:11:17.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:17.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:17.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:18.023 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:11:18.025 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi046/crash 2024-09-12T03:11:18.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:11:18.065 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:11:18.065 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:11:18.067 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi145/crash 2024-09-12T03:11:18.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:11:18.113 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:11:18.113 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:11:18.115 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi196/crash 2024-09-12T03:11:18.116 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:11:18.148 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:11:18.148 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:11:18.149 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:11:18.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:11:18.255 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:11:18.255 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:11:18.257 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:11:18.260 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:11:18.294 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:11:18.295 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:11:18.297 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:11:18.297 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:11:18.297 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:11:18.305 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:11:18.305 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:11:18.305 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:11:18.305 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:11:18.305 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:11:18.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:11:18.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.6.log/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.a.log: 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 87.8% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:11:18.311 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:11:18.313 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:11:18.313 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:11:18.313 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:11:18.313 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:11:18.313 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:11:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:11:18.359 INFO:teuthology.orchestra.run.smithi145.stderr: 81.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:11:18.394 INFO:teuthology.orchestra.run.smithi145.stderr: 92.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:11:18.415 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:11:18.415 INFO:teuthology.orchestra.run.smithi046.stderr: 84.1% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:11:18.443 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:11:18.455 INFO:teuthology.orchestra.run.smithi145.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:11:18.470 INFO:teuthology.orchestra.run.smithi145.stderr: 94.6% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:11:18.476 INFO:teuthology.orchestra.run.smithi145.stderr: 90.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:11:18.491 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:11:18.498 INFO:teuthology.orchestra.run.smithi046.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-12T03:11:18.504 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:11:18.511 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:11:18.513 INFO:teuthology.orchestra.run.smithi196.stderr: 81.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:11:18.529 INFO:teuthology.orchestra.run.smithi046.stderr: 94.7% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:11:18.535 INFO:teuthology.orchestra.run.smithi145.stderr: 87.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:11:18.556 INFO:teuthology.orchestra.run.smithi196.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:11:18.589 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:11:18.600 INFO:teuthology.orchestra.run.smithi145.stderr: 91.0% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:11:18.694 INFO:teuthology.orchestra.run.smithi046.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:11:18.714 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:11:20.095 INFO:teuthology.orchestra.run.smithi046.stderr: 90.2% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:11:20.143 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:11:20.489 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:11:20.546 INFO:teuthology.orchestra.run.smithi145.stderr: 93.5% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:11:22.625 INFO:teuthology.orchestra.run.smithi145.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:11:23.835 INFO:teuthology.orchestra.run.smithi196.stderr: 94.2% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:11:26.875 INFO:teuthology.orchestra.run.smithi196.stderr: 92.0% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:11:35.862 INFO:teuthology.orchestra.run.smithi046.stderr: 90.2% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:11:41.948 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:11:53.974 INFO:teuthology.orchestra.run.smithi196.stderr: 93.4% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:23:57.460 INFO:teuthology.orchestra.run.smithi145.stderr: 92.1% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:26:17.824 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:26:20.098 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:27:26.842 INFO:teuthology.orchestra.run.smithi145.stderr: 92.0% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:27:55.768 INFO:teuthology.orchestra.run.smithi145.stderr: 91.8% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:28:01.927 INFO:teuthology.orchestra.run.smithi196.stderr: 91.8% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:28:02.715 INFO:teuthology.orchestra.run.smithi145.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:28:02.727 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-12T03:28:02.727 INFO:teuthology.orchestra.run.smithi145.stderr:real 16m44.452s 2024-09-12T03:28:02.727 INFO:teuthology.orchestra.run.smithi145.stderr:user 7m15.473s 2024-09-12T03:28:02.727 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m19.167s 2024-09-12T03:29:24.913 INFO:teuthology.orchestra.run.smithi196.stderr: 91.7% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:29:24.915 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-12T03:29:24.915 INFO:teuthology.orchestra.run.smithi196.stderr:real 18m6.640s 2024-09-12T03:29:24.915 INFO:teuthology.orchestra.run.smithi196.stderr:user 8m19.815s 2024-09-12T03:29:24.915 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m21.635s 2024-09-12T03:37:56.384 INFO:teuthology.orchestra.run.smithi046.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:41:35.054 INFO:teuthology.orchestra.run.smithi046.stderr: 91.9% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:44:00.652 INFO:teuthology.orchestra.run.smithi046.stderr: 91.8% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:49:26.208 INFO:teuthology.orchestra.run.smithi046.stderr: 91.7% -- replaced with /var/log/ceph/2691fc0c-70b0-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:50:03.099 INFO:teuthology.orchestra.run.smithi046.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-12T03:50:03.101 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-12T03:50:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:real 38m44.829s 2024-09-12T03:50:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:user 12m37.273s 2024-09-12T03:50:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m35.726s 2024-09-12T03:50:03.102 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:50:03.104 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi046/log 2024-09-12T03:50:03.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:51:49.478 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi145/log 2024-09-12T03:51:49.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:53:01.430 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi196/log 2024-09-12T03:53:01.432 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:54:24.280 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:54:24.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force 2024-09-12T03:54:24.656 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:54:25.459 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force 2024-09-12T03:54:26.020 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:54:26.744 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2691fc0c-70b0-11ef-bcea-c7b262605968 --force 2024-09-12T03:54:27.071 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 2691fc0c-70b0-11ef-bcea-c7b262605968 2024-09-12T03:54:27.888 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:54:27.889 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:54:27.909 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:54:27.929 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:54:27.945 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:54:27.945 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:54:27.958 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:54:27.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:54:27.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:54:27.972 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:54:28.206 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:54:28.216 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:54:28.216 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:54:28.219 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:54:28.222 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:54:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:54:28.236 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:54:28.237 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:54:28.266 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:54:28.266 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-12T03:54:28.267 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -2592us[-2586us] +/- 41ms 2024-09-12T03:54:28.267 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 34 +1612us[+1618us] +/- 72ms 2024-09-12T03:54:28.267 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +2251us[+2257us] +/- 66ms 2024-09-12T03:54:28.267 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 -2528us[-2533us] +/- 41ms 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +1669us[+1663us] +/- 72ms 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +2295us[+2290us] +/- 66ms 2024-09-12T03:54:28.280 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:54:28.295 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:54:28.295 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-12T03:54:28.295 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 8 377 162 -2591us[-2558us] +/- 70ms 2024-09-12T03:54:28.295 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 +1605us[+1605us] +/- 72ms 2024-09-12T03:54:28.296 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 161 +2309us[+2309us] +/- 64ms 2024-09-12T03:54:28.296 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:54:28.296 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:54:28.307 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:54:28.307 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:54:28.317 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:54:28.467 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:54:28.622 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:54:28.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:54:28.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:28.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:54:28.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:28.815 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:54:28.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:28.843 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:54:28.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:54:28.863 INFO:teuthology.task.internal:Duration was 4860.593841 seconds 2024-09-12T03:54:28.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:54:28.874 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:54:28.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:54:28.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:54:28.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:54:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:54:28.950 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:54:28.955 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:54:29.281 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:54:29.281 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-12T03:54:29.281 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:54:29.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-12T03:54:29.310 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:54:29.342 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-12T03:54:29.342 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:54:29.367 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:54:29.367 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:54:29.370 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:54:29.385 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:54:29.434 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:54:29.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:54:29.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:54:29.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:54:30.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:54:30.024 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:54:30.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:54:30.054 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:54:30.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:54:30.109 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:54:30.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.156 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-12T03:54:30.161 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-12T03:54:30.182 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-12T03:54:30.219 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:30.234 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:30.249 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:54:30.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:54:30.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:54:30.284 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:54:30.286 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi046 2024-09-12T03:54:30.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:54:30.335 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi145 2024-09-12T03:54:30.335 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:54:30.374 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901632/remote/smithi196 2024-09-12T03:54:30.376 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:54:30.415 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:54:30.415 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:54:30.417 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:54:30.420 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:54:30.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:54:30.476 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:54:30.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:54:30.487 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:54:30.488 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:54:30.493 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:54:30.495 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:54:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:54 /home/ubuntu/cephtest 2024-09-12T03:54:30.508 INFO:teuthology.orchestra.run.smithi145.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:54 /home/ubuntu/cephtest 2024-09-12T03:54:30.519 INFO:teuthology.orchestra.run.smithi196.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:54 /home/ubuntu/cephtest 2024-09-12T03:54:30.520 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:54:30.532 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:54:30.563 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} duration: 4860.59384059906 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:54:30.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:54:30.724 INFO:teuthology.run:pass